2023-01-02
fixed in assp 2.6.7 *SPAM-Evaporator* build 23002:

- ASSP_AFC.pm is upgraded to version 5.45

- on several linux version assp throws an error "Error: Schedule entry '0-59/10 * * * *' for MemoryUsageCheckSchedule is not valid"
  this should no longer happen




2022-11-22
fixed in assp 2.6.7 *SPAM-Evaporator* build 22326:

- several interal statistics were unexpected cleared at startup

- if a mail with the empty envelope recipient was received and the sending domain (from: or sender:) provided a DMARC record
  the DMARC check failed. The DMARC check is now skipped for such mails.

changed:

- the top ten statistc URL is expanded with a count parameter

..../top10stats?count=10
  the number specifies the amount of the shown entries
  a negative value will show the less blocked entries
  
  the hidden variable $toptencount = 10; will set the default value


added:

- the 'work with addresses and domains' GUI-dialog now allow to request a blockreport in the web session.
  
  ... blockreport: or block: or report: or blr: or bl: in front of an address or in the reason field will generate
  a blockreport in a new browser window - a trailing number and/or regex specifys the days and filter....

  NOTICE: these blockreports are internaly executed with EmailAdmin equivalent permission (no restriction)

- the 'work with IP-addresses' GUI-dialog supports the same blockreport option like the work with addresses and domains' GUI-dialog
  this option is and will be keeped undocumented

- admin users action permissions are enhanced with the 'action parameter' "webblockreport"
  so even an admin user is allowed to work with the addresses dialogs, it is possible to disallow the blockreport generation in the browser

- the 'work with IP-addresses' GUI-dialog is now able resolve SPF-records - simply write SPF:domain.org in to the input field




2022-11-14
fixed in assp 2.6.7 *SPAM-Evaporator* build 22318:

- some spam messages were not forwarded even 'ccSpamAlways' was configured
  NOTICE: if 'ccSpamAlways' is configured and 'SpamVirusLog' is not set to 'quarantine' (even it is set to 'no collect' !!), the virus mails
          will be sent to the configured email address - GUI: ..."Copy Spam to these recipients regardless of collection mode."...
          this not a changed behavior - but keep this in mind!

- it was very hard for an admin to find out why a specific spam mail was not forwarded to sendAllSpam
  if SessionLog is set to verbose, the reasons are now logged to the maillog.txt

- some connections of already finished spam messages were running in to a SMTPTimeout - resulting in a very high count of SocketCalls
  for the connection

- using postfix as local backend server, it was possible that assp sent orphaned data to postfix, which caused postfix to respond with
  '502 5.x.x syntax error' replies
  this was happen, if assp ignored the socket read-error 'EAGAIN - resource temporary not available' in some very special cases

- mails from gmail.com or googlemail.com users who sent automatic generated mails to your assp, were rejected/scored by DoNoFromSelect
  
  a good example for this case are google-calendar invitations: the envelope sender is ...@calendar-server.bounces.google.com
  the from address is the right ..@gmail.com or ...@googlemail.com user address - the sender header addess is a ...@google.com address
  the missmatch of the domain names caused assp to score and/or reject the mail
  DoNoFromSelect now processes gmail.com, googlemail.com and google.com as equal domains (internaly all domains are set to gmail.com for the 
  DoNoFrom check)
  
- the statistic was not counted, if a DKIMidentityWLmatch or DKIMidentityNPmatch was found

  

2022-11-09
fixed in assp 2.6.7 *SPAM-Evaporator* build 22313:

- if any of DKIMWLAddresses or DKIMNPAddresses was used and a mail contained more than one DKIM signature (eg. for different identities)
  only the first (header - top to bottom) valid DKIM identity was checked against both parameters
  now all found valid DKIM identities are checked agains DKIMWLAddresses and/or DKIMNPAddresses.



2022-11-06
fixed in assp 2.6.7 *SPAM-Evaporator* build 22310:

- on non english linux installations some times the connection retry was not working


changed:

- in 'ccSpamInDomain' it is now possible to use the USERNAME literal
....  The literal USERNAME is replaced by the user part of the recipient. ....


- the minimum version of the module 'Schedule::Cron' is changed to 1.03

- if the "NWLI" directive is used in weighted regular expressions, the skipping reason is now shown in the maillog for real mail processing,
  (not in the analyzer)


added:

- URL's encoded in base64 using the ".atob" HTML statement are now detected, decoded and checked in URIBL

- the ASSP_AFC plugin (5.44) is able to detect native integrated Base64 encoded as well as javascript code in "text/html" and "image/svg+xml"
  attachments, if "exe-bin" is configured to be detected.
  The native Base64 parts are decoded and analyzed like every other attachment.


- the ASSP_AFC plugin (5.44) now also supports the following two blocking exceptions
 :JSHTML - HTML file with JavaScript or mouse driven HTML events (like: onmouseover, onmouseout, onfocus, onblure ...)
 :JSSVG - SVG images with JavaScript or mouse driven HTML events (like: onmouseover, onmouseout, onfocus, onblure ...)


2022-10-20
fixed in assp 2.6.7 *SPAM-Evaporator* build 22293:

- if BlockRepForwHost was configured and any of EmailBlockReportDomain, EmailBlockReport or BlockRepForwHost was changed, queued forwarding requests failed
  in case any of these vales is changed, the BlockReport-Forwarding queue is cleared now

changed:

- If delaying is enabled and DelayUseNetblocks is set to 'On' and the perl module 'NetAddr::IP::Lite' is installed and enabled, assp resolves the SPF record for the domain used in the 
  envelope sender address. If the connected IP-address is valid (in terms of the SPF-record) all IP-adresses and ranges from the SPF-record are hashed and the hash is used
  (instead of the the connected IP-address) for delaying.
  This new behavior makes sure, that another valid IP-address, that tries to deliver the same mail after the first valid address was delayed, will be not delayed.
  If no SPF-record is available, the SPF-record is invalid or the connected IP-address is not valid - the connected IP-address will be used for delaying (old behavior).
  Because it is expected that a system will try to connect again after it was delayed, the SPF-records are cached inside assp.
  The records are stored in assp/tmpDB/files/SPFRecCache.sav and refreshed by the MaintThread, if any TTL gets outdated. 



2022-10-07
fixed in assp 2.6.7 *SPAM-Evaporator* build 22280:

- after an upgrade of the perl module Schedule::Cron to version 1.03 the assp scheduler was no longer working





2022-10-06
fixed in assp 2.6.7 *SPAM-Evaporator* build 22279:

- griplist uploads and downloads were no longer working, because the sourceforge http server no longer accepts plain http transfer
  NOTICE: ALL older versions of assp will fail to upload to the griplist server!

- statistc uploads were no longer working, because assp used http - now https is used by assp
  NOTICE: ALL older versions of assp will fail to upload to the stats server!

- after talking to the sourceforge support team, an exception is made for assp the assp project at the sourgeforge web-server to accept plain https connections for a short
  range of time. It is strongly recommended to upgrade your assp installation to the latest version!
  If the exception is canceled at any time, all older versions of assp will be unable to use griplist and stats.

- in some cases assp failed to decode empty MIME-encoded content (=?UTF-8?Q??=) correctly

changed:

- assp generates now 2048 bit RSA keys (instead of 1024 bit) if no SSL-keys/certs are found at startup

- all sourgeforge.net related URL's are changed to use https instead of http


2022-09-09
fixed in assp 2.6.7 *SPAM-Evaporator* build 22252:

- if line continuation '\' was used in a regular expression file, the regex was no longer working like expected

- if line continuation '\' was used in a regular expression file, the analyzer has'nt shown the matching file and line


2022-09-08
fixed in assp 2.6.7 *SPAM-Evaporator* build 22251:

- some unexpected log lines about unresolveable IP-addresses were shown by assp (since build 22200)

- the attachment 'NoCheckIf' rule was not working, if the SPF-check or the DKIM-check was skipped because of any condition (noprocessing, whitelisting, ...)



changed:

- until know soft-hyphens (&shy; 0xAD , U+00AD) were replaced by normal hyphens (- , 0x2D , U+002D) for all text related tests and operations in assp -
  from now on, soft-hyphens are removed from all text parts



2022-07-28
fixed in assp 2.6.7 *SPAM-Evaporator* build 22209:

- If an EmailAdmin requested a resend of a blocked mail (for another person) and the blocked mail was originally sent to multiple envelope recipients,
  the mail was only resent to the first original envelope recipient, because only the first found 'X-Assp-Intended-For:' header was parsed
  for recipient addresses by assp.



2022-07-19
fixed in assp 2.6.7 *SPAM-Evaporator* build 22200:

- if 'runAsUser' was used on nix systems, it was possible that the file assp/tmpDB/files/SPFRecCache.sav was saved using a wrong owner (root) -
  which may be caused a permission error at the next start of assp
  
- if setFilePermOnStart was set - the permission for several files and folders were set twice at the next start

- if a FQDN was defined for relayHost, which resolved to multiple IP-addresses - and relayAuthUser/relayAuthPass (or AUTHrelayTable) was configured
  it was possible, that assp has'nt detected the connected host for authentication and skipped the AUTH command (outgoing mails were rejected by the relay host)




2022-07-06
fixed in assp 2.6.7 *SPAM-Evaporator* build 22187:

- it was possible to define a colon (:) in 'proxyuser' - this is no longer allowed - as stated in rfc7235

- the upload of Stats and Griplist to the sourceforge servers failed, if 'proxyserver' and 'proxyuser' and 'proxypass' were configured

- the connection to an IPv6 backend-server (MTA) failed, if the client connected to a plain (not SSL) assp-IPv6-listener and _INBOUND_:Port
  was defined for the related destination and a "destination routing table" for the connected local V6-IP was not existing 

- if the root password was forgotten and 'webAdminPassword' was set in assp.cfg to a value either starting with '45' or with a length of 13 characters,
  weblogin was no longer possible for root, even the right was used in the GUI
  if a new root password is set in the assp.cfg, the case: it is starting with 45 and it is 13 characters long, will disable root login

  NOTICE: if you (change) set the value for 'webAdminPassword' manually in the assp.cfg while assp is not running
          YOU WILL LOSE ALL 
     - encryped configuration parameters
     - encrypted configuration files and included files
     - config synchronization contexts
     - defined assp GUI-users
     - licenses granted exclusively for this assp instance
     - global penaltybox registrations



2022-06-14
fixed in assp 2.6.7 *SPAM-Evaporator* build 22165:

- all plugins failed to detect assp.pl versions with two digit version numbers
  all plugins are updated

- assp on perl 5.36.0 showed unexpected warnings at compile time

- SSL renegotiations at a tranparent proxy connection caused SMTP-timeout on some systems



changed:

- perl version 5.36.x is now supported and shown as recommended perl version for assp

- to prevent inconsitent version checks, assp.pl provides the version-check code for all plugins

- 'maxSSLRenegotiations' is no longer checked for transparent proxy connections




2022-05-05
fixed in assp 2.6.7 *SPAM-Evaporator* build 22125:

- TLSv1.3 connections from assp to a backend-server were running in to a SMTP-timeout. This was caused by an unhandled second session-ID transmission in TLSv1.3.

 
changed:

- 'forceTLSIP' can now be configured for selected sender addresses/domains

- per default assp generates the 'X-Original-Authentication-Results' header line
  if the hidden config-variable 'genXOrigAuthResHeader' is set to zero, assp will generate the 'Original-Authentication-Results' header line instead.

- the connection-timeout-debug (ConTimeOutDebug) output is enhanced



2022-03-21
fixed in assp 2.6.7 *SPAM-Evaporator* build 22080:


- if 'myGreeting' was configured as multiline greeting, assp has prepended '220 ' even the first line was starting with '220-'



changed:

The literal 'LASTCOMMAND' will be replaced by the last used SMTP-command in every SMTP error reply.
The literal 'MAILFROM' will be replaced by received envelope sender in every SMTP error reply.
The literal 'RECEIVEDHELO' will be replaced by the received HELO/EHLO string in every SMTP error reply.


added:

'forceTLSIP','Force these IP's to use TLS*'
  Enter IP's that you want to be enforced to use SSL/TLS, separated by pipes (|).
  DoTLS needs to be set to "do TLS" to make this feature working!
  If a host or client uses the MAIL FROM: command without it used STARTTLS before or STARTTLS has failed or it is not connected to a SSL-listener 
  (the connection is not transport layer secured), the permanent SMTP-error code
  502 <MYNAME> connected by 'IPCONNECTED' - 'RECEIVEDHELO'. The used command 'LASTCOMMAND: <MAILFROM>' is still not supported, because the connection is NOT secured by an encryption layer (TLS) - please use STARTTLS first FORCEEXPLAIN
  will be sent by assp and the connection will be dropped.
  IP's listed in noTLSIP , private IP-ranges , IP's in SSL-failed-Cache and IP's connected to a NoTLSlistenPorts are excluded from being forced by this feature.
  To force all IP's, enter 0.0.0.0/0|0::0/0 .
  Mails to BounceSenders are also excluded from being forced by this feature! So TLSRPTv1 reports and other notifications are delivered, even TLS/SSL is in an invalid state.
  If a connection is dropped by this feature, the connected IP will get no penalty (score)!
  
  If this feature is enabled for all connecting IP's, it is highly recommended to configure MTA-STS (SMTP MTA Strict Transport Security - RFC 8461) or the more secure
  DANE (DNS-Based Authentication of Named Entities - RFC 6698, 7671)(SMTP Security via Opportunistic DNS-Based Authentication of Named Entities (DANE) Transport Layer Security (TLS) - RFC 7672)
  for your hosted domains!
  Notice: MTA-STS and DANE require both the SSL_version TLSv1_2 and/or TLSv1_3.



2022-03-16
fixed in assp 2.6.7 *SPAM-Evaporator* build 22075:

- if a DNS-TXT record contained more than one entry (multi line), only the first entry was read
  some SPF-records were not processed correctly for this reason
  this is fixed
  
  
changed:

- the GUI dialog "working with IP-addresses" supports now the calculation of resulting IP-networks, if an IP-address/range is removed from or added to an IP-range
  example: 192.168.0.0/16-192.168.1.0/24  or  192.168.0.0/24+192.168.1.0/24

- the 'SPF:' SPF-record definition for Groups and IP-lists is enhanced
  it is now possible to exclude SPF-include, SPF-redirect or IP-addresses/ranges from resolved SPF-records
  example: SPF:amazon.com -amazonses.com
  please read the general GUI-help or the Groups-GUI-help for the detailed explanation



2022-03-04
fixed in assp 2.6.7 *SPAM-Evaporator* build 22063:


added:

- The XOAUTH2 authentication mechanism is implemented (IN and OUT/relay). A SSL/TLS protected connection is required in every case for XOAUTH2 - independend from the setting of AUTHrequireTLS.
  The help text for AUTHrequireTLS, relayAuthUser and relayAuthPass is changed.
  To provide the XOAUTH2 authentication mechanism, the assp library module Authen::SASL::Perl::XOAUTH2 is required (assp/lib/Authen/SASL/Perl/XOAUTH2.pm). 

changed:

- transparentRecipients: 
   - moving to a transparent proxy connection requires now that all envelope recipients are matching transparentRecipients (not only one)
   - moving to a transparent proxy connection is now done after the DATA command is received - no longer when the RCPT TO: is received
  These changes are done for security reasons. There is no longer a chance to abuse a transparent connection for NON-transparentRecipients or not local recipients!


- the default value of 'MaxAllowedDups' is changed from 50 to 0 - the help text is changed

- the (minimum) recommended version for the perl module Email::MIME is changed from 1.946 to 1.950

- SignalLog is improved: if an unexpected signal is detected (like SEGV) and SignalLog is set to verbose, the complete perl caller stack is now written to the file debugSignal.txt




2022-02-27
fixed in assp 2.6.7 *SPAM-Evaporator* build 22058:

- because of a MIME decoding BUG, reported mails with very long subjects were not processed correctly

- if a blocked mail eml-file was moved from the spam folder to the discarded folder, it was sometimes no longer found by the BlockReport feature - and so no resendlinks were provided in the blockreports


added:

'send250toIP','Send 250 OK to this list of IP-addresses*',
 List of connecting IP-addresses which will get the reply '250 OK' instead of SMTP error codes ('5xx a.b.c') - see send250OK .
 This is a usefull setting, if a blocked sending host got a 5xx reply and does not follow the SMTP-RFC's (stop and send a NDR). Instead the host permanently tries to send the same mail again and again.
 Such blocked mails are internaly processed like any other SPAM mail, but the sender will not get informed about, that the mail was not delivered to the final recipient!


changed:

- if SPF: lists are included in to IP-address-list, the resolving of the SPF-records in now done in backgound and the results are cached for the lowest received TTL

- 'bombDataRe' (contrary to all other bomb-RE's) was running against the HTML undecoded body content only -
  now the HTML decoded and HTML undecoded body is checked (only the HTML line endings (=CRLF) are removed from the HTML undecoded body)

- the regular expression optimization is now disabled for a regex configuration parameter, if weights are used for it
  this way the definiton order for the regular expressions and their weights is keeped

- the processing speed for IP-address regular expressions is improved 

- ASSP_FC version 5.38 is released. The virus detection is enhanced to detect: https://www.bleepingcomputer.com/news/security/malicious-csv-text-files-used-to-install-bazarbackdoor-malware/





2022-01-19
fixed in assp 2.6.6 *SPAM-Evaporator* build 22019:

- If the used perl version was compiled without linking to libcypto, the perl 'crypt' command was without function. This caused the assp internal encryption engine to fail
  and all encrypted configuration values and files were unuseable. The password for the root user was not stored.
  Now, if such a perl version is found by assp, it will try to load the module Crypt::UnixCrypt, which has the same function like the perl internal crypt command. If this module
  can't be loaded, assp will die and shows a related hint at the command line.

- ASSP contains code to handle unexpected SEGV signal errors. The past has shown, that recovering assp to a normal state after a SEGV occured is impossible.
  Most times the maillog.txt was filled with thousands or even million of error lines.
  For this reason, assp will now try a restart, if a SEGV happens - if a restart is not possible, the assp process will be ended.


- If a query string in RBL-, RWL- and URIBL-queries was longer than 62 byte, the query was not processed by assp. The length of such a query string is now limited to 253 byte.
  The length of the labels in a domain string are limited to 63 byte.


changed:

- It was possible for years now (but undocumented) to provide api keys for RBLServiceProvider and URIBLServiceProvider.
  The documentation for both parameters is extended.
  ...  It can be possible, that you need to provide a privat key or ID in the query string for a URIBL Service Provider - like: your-key.query-data.uribl-provider.org
       In this case, define the URIBL Service Provider like: your-key.$DATA$.uribl-provider.org
       The string $DATA$ will be replaced by the queried data in each request.


- A new function is implemented in to all IP-address lists. It is now possible to include all IP's of a SPF-record of a domain in to IP-address lists.
  The help text is extended:
  .... For several IP-address lists in assp, it can be advantageous to include all IP's (and ranges) listed in the SPF-record of a specific domain (for example in noPB, noHelo, whiteListedIPs, ...).
       To provide this, simply write SPF: in front of the domain name in a list entry - like 182.82.10.0/24|SPF:amazon.com|2201:1::1 .
       In this example assp will replace the term SPF:amazon.com with the list of all IP's and resolved IP's defined in the SPF-record of amazon.com.
       This will also work for IP lists in a group definition. Assignments made to such an entry - like SPF:amazon.com=>[usergroup] will be added to each resolved SPF-IP-address.




2021-12-17
fixed in assp 2.6.6 *SPAM-Evaporator* build 21351:

- if 'onlyAUTHHeloRe' was used, a logline for a match was written after each SMTP command - now the match is shown only after the HELO/EHLO command was used


changed:

- ASSP_AFC.pm version 5.37 is now able to detect template injection in RTF-documents
  https://www.proofpoint.com/uk/blog/threat-insight/injection-new-black-novel-rtf-template-inject-technique-poised-widespread


- the status of the perl module IO::Socket::INET6 was set to deprecated by the cpan maintainer
  assp.pl uses now IO::Socket::IP for IPv6 handling
  ASSP_DCC.pl 2.02 also uses IO::Socket::IP for IPv6 handling
  IO::Socket::INET6 is no longer installed by the assp perl module installer (2.10)


- the module Mail::SPF::Query (SPFv1) is removed from assp.pl (only Mail::SPF is used)
  Mail::SPF::Query is no longer installed by the assp perl module installer (2.10)
  'LocalPolicySPF' is removed from configuration, it was only used by Mail::SPF::Query (SPFv1)
  'SPF2' is removed from configuration - it is no longer required, Mail::SPF (SPFv2) is permanently used





2021-11-24
fixed in assp 2.6.6 *SPAM-Evaporator* build 21328:


- If 'AddRWLHeader' was enabled and a RWLCache hit was found, no RWL-header was added to the mail.


changed:

- text/plain parts of a mail are now also cleanedup from (badly) added HTML-tags, which improves all text based features, because the text/plain parts are most times processed first

- ASSP_OCR.pm version 2.25 is released
  It implements a short time result cache, to prevent processing the same MIME-parts multiple times because the same mail was sent to multiple recipients.

- The clamav and the filescan feature are implementing a short time result cache, to prevent processing the same MIME-parts multiple times because the same mail was sent to multiple recipients.

- In rare cases it was possible to overload assp by sending a large mail to many recipients.
  This can now prevented by configuring the hidden parameter 'maxSMTPipRelaySessions' - if used, it should be set one less than the configured number of SMTP-Workers (NumComWorkers)
  
# (number) limit the connection count per IP for relay - 0 and noMaxSMTPSessions disables the check
our $maxSMTPipRelaySessions = 0;

- the RWLCache was used without giving any cached results in the maillog.txt - this caused confusion, if the log wére analyzed
- the term 'whitelisted' is replaced by 'trusted' in RWL-headers and log lines - this caused confusion, if the mails was not whitelisted because of lower RWL-trust value

- The RBL/DNSBL check was skipped, if a RWL-trust value of 2 was reached. From now, the RBL/DNSBL check is skipped, if a RWL-trust value of 2 is reached and RBLWL is not set.



2021-11-13
fixed in assp 2.6.6 *SPAM-Evaporator* build 21317:

- If a line in a regular expression file was protected from regex optimization using the <<<...>>> pagma, a possibly defined weight (...=>ddd) was ignored and the default penalty points were used.

- If a predefinition of regular expressions like '(?(DEFINE)(?<..>...))' was used, assp has destroyed it sometimes to (?(?:DEFINE)...)


changed:

- files used in configuration parameters (file:...) are now supporting line continuation by adding a backslash '\' at the end of a line

- SMTP-replies (554 ...Service denied ...) sent because a connection is terminated very early (before HELO) are now extended to 554 ...Service denied for IP x.x.x.x , ...
  where x.x.x.x contains the connected IP address. This provides better backtracking of early blocked connections.



2021-11-02
fixed in assp 2.6.6 *SPAM-Evaporator* build 21306:

- An email, starting with an invalid MIME header, sent to the analyzer - prevented the analyzer from finding the subject of the mail.

- If the same regular expression was used with different weights, tagged by different 'NWLI' extensions, the regex itself was defined and executed multiple times.


changed:

- GUI help changed for
AddDKIMHeader
EmailSpam
EmailHam
NWLI

- If ReportLog is set to diagnostic and a .msg (outlook OLE) file is reported, which starts with invalid header content, this invalid content will be corrected and the original .msg file
  and the converted .eml file are stored in the assp/debug folder. So possible problems with outlook reporting can be better solved.
  .msg files can be converted to the MIME content (.eml) using the following command.
  
  Linux/nix: perl -e 'use Email::Outlook::Message; print Email::Outlook::Message->new(q(FILENAME))->to_email_mime->as_string;'
  Windows: perl -e "use Email::Outlook::Message; print Email::Outlook::Message->new(q(FILENAME))->to_email_mime->as_string;"

  Replace FILNAME with the name of the .msg file. The MIME content is written to STDOUT - ' >file.eml' at the end of the command will write the MIME content to an output file.



2021-10-29
fixed in assp 2.6.6 *SPAM-Evaporator* build 21302:

- Improved email address detection in the emailinterface list reports (whitelistadd , whitelistremove, ....).

- The change time for include files used in the 'Groups' feature were not recorded in workers. This caused unexpected configuration reloads in the workers, until
  assp was restarted.

- Any change made for 'Groups' caused a reload for all configuration parameters where a group was used, even the related group was not changed. A configuration reload is now
  only done for changed groups and there related configuration parameters.

- Unexpected results were produced by the analyzer, if emails were sent as (not zipped) attachment to the emailinterface for analyzing - using outlook as mail client (+exchange).
  Notice: always compress (e.g. zip) reported emails before they are sent to assp!


changed:

- If the hidden parameter 'DoRBWhite' is set, the rebuildspamdb process searches for matches in
  whiteRe, npRe, whiteListedDomains, noProcessingDomains, whiteListedIPs, noProcessingIPs, DKIMWLAddresses and DKIMNPAddresses -
  and removes those mails from the assp/spam folder. 


- 'ReportLog','Enable Report logging'
  'If set to diagnostic, each received report mail will be stored in the assp/debug folder.'
   This makes more easy to track down report problems, based on the data sent by the mail client to assp.

- The GUI description for the NWLI enhancement (for regular expressions) was updated. The code was changed to get the NWLI results exactly like descriped in the GUI.

- A hint (and context help) about encryped configuration parameters and files was added to GUI.


added:

The set hidden parameter 
DoRBBlack = 0;                      # (0/1) check blacklisted mails on rebuildspamdb (default 0 - 1 = skip rebuild for notspam if black)
removes all mails in the assp/notspam folder, which matches  :  noBlockingIPs, denySMTPConnectionsFromAlways, denySMTPConnectionsFrom and blackListedDomains

Notice: if all of DoRBWhite, DoRBBlack and DoRBRed are enabled, the rebuild process will take ~12 times (or very much) longer than without setting these switches.
        Don't be confused. If .eml files were corrected by spam/ham reports, assp will process them correctly. But it may help to maintain the corpus from time to time.




2021-10-20
fixed in assp 2.6.6 *SPAM-Evaporator* build 21293:

- if a file for regular expressions contained an incomplete default definition for the !!!NWLI!!! directive, this directive was not applied to the regexes in the file



changed:

- some corrections and additions to the main help text in the GUI

- the behavior of the 'NWLI' extension in regular expression definitions is enhanced

The NWLI conditions defined in a line are combined using a logical AND -- so N-W+ is combined to: NOT noprocessing AND whitelisted.
In fact, the weight is skipped, if any of the defined NWLI options does not match for a mail. If multiple lines would match, the weight of the first matching line is used.
This way you can define different weights for the same regular expression, but different mail states like in this example:
(1) foo=>0:>NW - weight is zero if noprocessing AND whitelisted
(2) foo=>0.5:>NW- - weight factor is 0.5 if noprocessing AND NOT whitelisted
(3) foo=>1.5:>N-W - weight factor is 1.5 if NOT noprocessing AND whitelisted
(4) foo=>55:>N-W- - weight is 55 if NOT noprocessing AND NOT whitelisted
(5) foo=>2:>W - this line will not be processed, because line 1 or 3 would have matched before, depending on the noprocessing flag
(6) foo=>2:>N- - this line will not be processed, because line 3 or 4 would have matched before, depending on the whitelisted flag

 

2021-10-17
fixed in assp 2.6.6 *SPAM-Evaporator* build 21290:


- build 21287 caused an error 'BerkeleyDB-ERROR: in start rebuildAddCorrections - syntax error at (eval 679) line 5, near "$main::$BDBerrLog "'

- fixes a STATS counting mistake since 21280

- if an IP was blocked by an early blocking feature (like maxAUTHError) at least 'maxSMTPipSessions' times, this IP was blocked by 'maxSMTPipSessions' until assp was restarted



2021-10-14
fixed in assp 2.6.6 *SPAM-Evaporator* build 21287:

- If a folder was defined for the parameter 'griplist' (e.g. grip/griplist) and this folder was not extisting, all griplist functions were not working.
  If a folder is now defined, it is created by assp.

- If 'ConfigChangeSchedule' was used to change a hidden configuration parameter, only the main thread (not any worker) was aware of the change.

- If a mail subject contained a questionmark '?' in its text and the subject header line was encoded 'Quoted Printable' and the questionmark was not right MIME encoded
  (instead it was written as '?') all internal functions related to the mail subject were not working correctly


changed:

- BerkeleyDB error logs (BDB-error.txt) are no longer permanently created and locked
  Instead there is a new hidden parameter 'BDBerrLog', which can be set to 1 to monitor BDB-problems.

our $BDBerrLog = 0; # (0/1) log BerkeleyDB errors in the related BDB-ENV -errfile .../BDB-error.txt (default = 0)

- The GUI-help text for 'noGriplistUpload', 'noGriplistDownload' and 'gripValencePB' are updated - griplist functions are not changed



added:
- If windows systems are running out of available open file descriptors and the used perl installation is not compiled using the -DUSE_PERLIO switch,
  the following parameters can be used to increase the available file descriptors for the assp process

our $winSetMaxIO_DLL = 'msvcrt';         # the name of the microsoft C-runtime-library used by perl and/or perl-modules (Win32 only !!!) - default is msvcrt
                                         # If your perl uses (is compiled against) any other msvcrtXXX (for example: msvcrt160 or msvcrt100) - change this value, if
                                         # you want to set the maximum open files limit in the msvcrtXXX.
                                         # This value is ONLY used for the below purpose ($winSetMaxIO), it has no other effect !

our $winSetMaxIO = 0;                    # (0/1/ 512 * 2**N) set the maximum open files limit (Win32 only !!!) in ($winSetMaxIO_DLL) msvcrt.dll (_getmaxstdio , _setmaxstdio)
                                         # 0 - use the default setting in msvcrt.dll (normaly set to 512)
                                         # 1 - find the maximum allowed value between 512 and 8192 and set it
                                         # 512 * 2**N - try to set the value as high as possible up to the given maximum (min 512 , max 8192, in 512 * 2**N [N=0..4])
                                         #          if the defined value is less than the current maximum, the setting will not be changed
                                         # Notice: PERLIO (perl compiled with -DUSE_PERLIO - check with :>perl -V) may define a different max open file limit for its
                                         #         IO's (defaults to 2048 because PERLIO_MAX_REFCOUNTABLE_FD=2048)
                                         #         - this limit is not affected by this value


2021-10-07
fixed in assp 2.6.6 *SPAM-Evaporator* build 21280:


- if $fakeAUTHsuccess was set, the collected .eml files contained only the X-Assp headers - not the spam mail data

- the definition of an invalid regular expression in 'NotifyRe' may caused a crash of the assp process




changed:

- The default value for 

$ignoreEarlySSLClientHelo
# (0/1) 1 - unexpected early SSLv23/TLS handshake Client-Helo-Frames are ignored , 0 - unexpected early SSLv23/TLS handshake Client-Helo-Frames are NOT ignored and the connection will be closed 

is changed from 1 to 0 in assp.pl

to recover the old setting, you may change assp.pl or you can set $main::ignoreEarlySSLClientHelo = 1; in lib/CorrectASSPcfg.pm sub set{}
The setting '1' was used as default to ignore early SSL connections from local clients.
 

- mails which are catched by 'fakeAUTHsuccess' are now counted for the statistics in STATS:msgMaxErrors and SCORESTATS:MaxErrors




2021-10-04
fixed in assp 2.6.6 *SPAM-Evaporator* build 21277:

- If a relational DB engine was used for hmmDB and/or SpamDB, the start of the worker 10001 (rebuildspamdb worker) has taken much more time (under certain conditions)
  compared to the start time of all other workers

- Improved error handling in case a client or server connects to the default SMTP-listener (25) using SSL.

- The 'fakeAUTHsuccess' feature was not working like expected (caused by a fix for AUTH-error handling in assp 2.6.4 *SPAM-Evaporator* build 19284).


added:

- At the point in time the X-ASSP-..  headers were calculated, assp now calls '&CorrectASSPcfg::modMyHeader($Con{fh})' (if this sub is defined).
  This makes it possible to in place modify $Con{fh}->{myheader} to any special needs.
  Keep in mind: Modifying SMTP-headers in a wrong manner may prevent mails from beeing transported!



2021-08-06
fixed in assp 2.6.6 *SPAM-Evaporator* build 21218:

- the SMTP extension PIPE_CONNECT is removed from the EHLO answer and is no longer a valid SMTP command



2021-07-21
fixed in assp 2.6.6 *SPAM-Evaporator* build 21202:


added:

# SSL/TLS assp will try to do a fast bulk (boost) write using the following values
# the max framesize of SSL is 16384 byte - for the time given in SSL_write_boost_max_time, assp will try to send as much SSL frames as possible
# be carefull changing any of these values !
our $SSL_write_boost = 0;           # (0/1/2) 0 - disabled - possibly auto enabled   (default)
                                    #         1 - enabled  - possibly auto disabled
                                    #         2 - permanently enabled (never automatically changed)
                                    # try the SSL boost - automatically set to 1 in ConfigChangeTCPBuf if $maxTCPSNDbufSSL > 16384
                                    #                     automatically set to 0 in ConfigChangeTCPBuf if $maxTCPSNDbufSSL <= 16384

our $SSL_write_boost_max_time = 100;# time in milliseconds used for SSL write boost (default 100)


changed:

'TCPBufferSize','TCP and SSL Read/Write Buffer Size'
  Define the buffer size in byte used for TCP- and SSL socket read and write operations - defaults to empty.
  Any or all of the following four values can be defined:
  
  tcprcv - TCP receive buffer size
  tcpsnd - TCP send buffer size
  sslrcv - SSL receive buffer size
  sslsnd - SSL send buffer size
  
  SSL-Write-Boost (currently: 1)
  
  Multiple value definition have to be separated by comma or pipe, like: tcprcv = 65536, tcpsnd = 65536, ...
  Possible size values are 8192 (8KB) to 99999999 (~95MB), special value for sslrcv and sslsnd is zero.
  Do NOT write dots in to the number values - like tcprcv = 1.048.576 , those values are not accepted!
  If a value is not specified for tcprcv, the TCP receive buffer size reported by the system is used - but at least 8192 byte.
  If a value is not specified for tcpsnd, the value is set to 99999999.
  If a value is not specified for sslrcv or sslsnd, a value of 16384 byte is used, which is the maximum size of a single SSL frame of the SSL layer.
  If a value of zero is specified for sslrcv or sslsnd, the according TCP socket buffer size is used.
  If the configured or calculated value for sslsnd is larger than 16384, the buffer for sslsnd is set to the (SSL)-maximum of 16384 byte and SSL-Write-Boost will be enabled.
  If SSL-Write-Boost is enabled assp sends for a maximum of 100 milliseconds in each SSL/TLS-connection as much data as possible in a single thread loop.
  Under normal conditions any setting here is not required - or better, is at least safe for all operating systems.
  But, if you notice a too low transmission speed (eg. for large mails) of plan TCP-sockets or SSL-sockets, it may help to set (increase) the according buffer values.
  like: tcprcv = 1048576, tcpsnd = 10485760, sslrcv = 0, sslsnd = 0
  To monitor your settings, set SessionLog to diagnostic and watch the maillog.
  Notice: setting any receive buffers too high may cause the operating system to fall back to very low values (eg. 8KB), which will slow down the transmission speed dramatically.
  On most systems the TCP send buffers size can safely be set to the maximum supported value of 99999999 (tcprcv default).




2021-07-17
fixed in assp 2.6.6 *SPAM-Evaporator* build 21198:

- the '^° and '$' were not working in 'invalidHeloRe'

- invalid UTF8 characters in a mail subject may caused SEGV errors in 'Win32::Unicode::Dir' on windows systems



2021-06-17
fixed in assp 2.6.6 *SPAM-Evaporator* build 21168:

- SMTP workers were dieing on spam mails from outlook.com because of a wrong DKIM signature (without an identity)


2021-06-16
fixed in assp 2.6.6 *SPAM-Evaporator* build 21167:

- Downloaded Griplist-IPv6 addresses were not correctly merged in to the local griplist.
- Under certain conditions the local griplist was cleared at startup, which caused a full download of the griplist.
- The BerkeleyDB engine has shown errors, even there was no BerkeleyDB-Error condition.
- Gripvalues were not shown (or not correcty shown) in the analyzer
- Blockreports initiated by an admin or schedule and send to a regular user unexpected contained links to open the blocked mail in the assp GUI.
- If 'ispgripvalue' was cleaned (set to empty), there was no related calculated value (x) in the Griplist available.

changed:

- IPBlocking and IPExtremeBlocking are now separated checks - but there are no functional changes.
- If SessionLog or ConnectionLog are set to at least verbose and a check is skipped for a mail, this is now shown in the maillog.txt.

- ASSP_AFC.pm version 5.32 improves the detection of error conditions in libarchive (fatalerror -25), where an archive entry can't be extracted because of required but
  unsupported extraction methodes. 


2021-05-27
fixed in assp 2.6.6 *SPAM-Evaporator* build 21147:

- All links in Blockreports are containing the HTML directive 'target="_blank"' to open the link in a new window.
  This causes problems in some Web-Mail-Clients to provide the resend request mail.
  The hidden variable 

our $TargetBlank = {                     # where to include 'target="_blank"' in to HTML links - set the value to '' if 'target="_blank"' makes problems
    'BlockReport' => ' target="_blank"'  # BlockReports in WebMail-Clients like thunderbird, Roundcube Webmail and possibly others will need to set this to '' to make the resendlinks working
};

  can be used, to skip this directive in the links.
  
  Another possible solution is to remove the directive from the html part using lib/BlockReport/modify.pm by adding the following
  line to the %toReplace hash in sub modify - like:

&makeRe('target=3D"_blank"') =>  '', 


- IPv6 addresses were not reported correctly to the Griplist-Server and because of this, these addresses were not provided by griplist downloads.

- If database drivers others than 'mysql' or 'MariaDB' were used, assp warned about the missing driver function '_async_check'.
  The warning is no longer logged in this case.

- Resent mails were sometimes rejected by local MTA's, if assp is running on nix systems. This was caused by wrong line endings ([LF] only)
  in the resent mails, which were not corrected by assp.


changed:

- '+' signs in mail subjects caused problems in Blockreports in Web-Mail-Clients. The file name of the file to resend was wrong parsed
  and the '+' signs were replaced by spaces. Because of this, assp was unable to find the file in the filesystem.
  '+' signs are no longer used in (eml) file names to prevent this.


added:

'addErrorReplyExplanation','Add an Error-Reply-Explanation'
'The text defined here will be added to every permanent SMTP-error-reply (starting with 5xx - except 500, 501, 502, 503, 504, 521, 534, 535, 538).
 For example to add a web link, where blocking reasons are explained.
e.g.:
- error explanations at https://your.web.domain/block-reasons
or
- error explanations at https://your.web.domain/block-reasons?session=SESSIONID&amp;ip=IPCONNECTED

The text (and possibly a clickable link) will become visible to blocked senders in the NDR (No Delivery Report) of the blocked mail.
In the second example the assp session-id and the connected IP-address are part of the link. The web server can extract the log entries
for the mail from the maillog.txt and can explain much better and/or check the database for the IP reputation and ... and ...  .
If you want to skip this addition for any configurable SMTP-reply, write the literal NOEXPLAIN at the end of the configured SMTP-reply definition.
To force the addition for any of the above shown exceptions, add the literal FORCEEXPLAIN to the reply definition.
Both literals will be removed from the reply before it is sent.
For example, to skip the addition in SpamError: 554 5.7.1 Mail appears to be unsolicited -- send error reports to postmaster@LOCALDOMAIN NOEXPLAIN
Keep in mind, that the maximum length of a complete SMTP reply line should not exceed 512 byte (XXX text [CR][LF])



2021-04-12
fixed in assp 2.6.6 *SPAM-Evaporator* build 21102:

- If the last line of a bomb regular expression file was a comment (#), assp has warned that the regex matches an 'empty' string
  and 'all' strings - because the resulting regex ended with a pipe (|)

- The rebuild-spamdb-task failed, if MSSQL was used as backend-DB for SpamDB and/or HMMDB - because the standard ANSI-SQL statement
  to rename the temporary table is not supported by MSSQL. If MSSQL is detected by assp, the right SQL-satement is now used (sp_rename).

changed:

- Several CIDR-perl modules (also Net::CIDR::Lite 0.22) changed their behavior. IP-addresses with leading zeros in IPv4-address octets are no
  longer allowed (for example: 010.072.100.008 has to be defined as 10.72.100.8). Please have a look in to your manually edited IP lists
  and correct those cases. Code is added to autocorrect those IP-addresses when IP-lists are read. But there are may be circumstances, when this
  autocorrection will not take place!
  A related hint in red color is added to the bottom of the GUI.


2021-03-15
fixed in assp 2.6.6 *SPAM-Evaporator* build 21074:

- The definition in 'myGreeting' has only replaced the last greeting line (220 text) from the MTA, lines starting with 220-text
  were not touched. Now all greeting lines are replaced.

- Under rare conditions it was possible, that the penaltybox black and white contained wrong formated entries. Such entries are
  now removed from these lists.


2021-02-21
fixed in assp 2.6.4 *SPAM-Evaporator* build 21052:


changed:

- mails to local postmaster@ and webmaster@ are no longer hardcoded skipped from backscatter checks
  if you want skip them in future from backscatter checks, add the addresses to 'noBackSctrAddresses'


- The SPF check fails if the sender is not RFC822/RFC1522 conform. In this case the SPF result got the tag ...(cache), which was confusing if the SPF cache was disabled.
  Now SPF check also fails for this case, but the added tag is now ...(RFC822)
  

- ASSP_AFC.pm version 5.29 is at least recommended

- ASSP_OCR.pm version 2.24 is at least recommended




2020-12-29
fixed in assp 2.6.4 *SPAM-Evaporator* build 20364:

- building groups using LDAP failed for users, if the CN= of a user contained a comma or another unexpected unescaped character

- the internal database locking was skipped, if MariaDB was used instead of MySQL

- the perl module status screen reported a missing VirusTotal-Key even a valid key was configured

- sending reports failed in some cases with a syntax-error in MAIL FROM:

- some times BlockReports contained two date: header lines, which caused some MTA's to reject the report

- replacing the assp.cfg while assp was running caused unexpected warnings in the maillog.txt, even the new configuration was loaded without a mistake


changed:

- ASSP_AFC 5.26 is available: it improves the detection of obfuscated executable code in PDF-files
  fix: under rare conditions some attachments were not analysed because of an internal UTF8 error  



2020-11-05
fixed in assp 2.6.4 *SPAM-Evaporator* build 20310:

- trailing digits in the hostname (like 'mx.microsoft.com 1') in ARC-header lines were leading in to a 'notmatch' for trusted forwarer definitions



changed:

- The rebuildspamdb.pm module is upgraded to version 8.03. It provides faster rebuild processing, and much shorter locking times for HMMdb and SpamDB.

- performance improvement for the import/export database feature

- if email addresses and IP-addresses are managed using the GUI, a given reason and the date are written to the comment of the modified line

- improved MIME-header fixup for missing boundary definitions

- improved database cache handling


added:

'RebuildUsesFileModel','Build a Model from all processed emails for faster processing'

 The rebuild task builds a content model (in memory or BerkelyDB only) of all processed files, and uses this model at the next rebuild for faster processing.
 The time to process the mail-files is reduced down to a tenth (if BerkeleyDB is not used ( useDB4Rebuild OFF )), but requires a large amount of additional memory - eg. 2GB.
 The time to process the mail-files is reduced to a half, if BerkeleyDB is used ( useDB4Rebuild ON ).
 The default setting is ON
 The first rebuild after setting this to ON will run at a normal speed - all the next rebuild tasks will run faster.




2020-10-09
fixed in assp 2.6.4 *SPAM-Evaporator* build 20283:

- fixes a vtapi exception if the configured proxy is not available


2020-10-06
fixed in assp 2.6.4 *SPAM-Evaporator* build 20280:

- whoisIP - queries were not working if ASSP_Selfloader was disabled

- the noprocessing flag caused by ip-addresses and/or email-addresses/domains was wrong set to a lower priority, if 'npSize' was reached

- the VirusTotal API was not working if a proxy server was defined but no proxyuser

- if the DKIM-Identity was not set in the DKIM-configuration, the identiy was calculated without the leading '@' (and possibly rjected by the receiver)

- the receipient-replacement feature returned the orignal email address instead of the expected replacement, if a match was found but the next jump target rule has not matched



changed:

- Perl 5.32 is now supported

- ASN-Provider can now be configured in hidden variables
our $ASNProviderIPv4 = '.asn.routeviews.org |
                        .origin.asn.cymru.com';  # asn.routeviews.org equivalent provider for IPv4 ASN, if local (or others) DNS provides the ASN list
                                                 # combine multiple providers (for failover) by pipe '|' or comma ','
our $ASNProviderIPv6 = '.origin6.asn.cymru.com'; # asn.routeviews.org equivalent provider for IPv6, if local (or others) DNS provides the ASN list
                                                 # combine multiple providers (for failover) by pipe '|' or comma ','

- all IANA-reserved IPv4 and IPv6 addresses are now treaded as private IP-addresses (igored in IP checks)

- The free VirusTotal queries for GPB-Clients are now restricted to 4096 per day

- the origin ip-address detection is improved (eg. 255.255.255.255 used by some cloud providers, caused mistakes)




2020-08-11
fixed in assp 2.6.4 *SPAM-Evaporator* build 20224:

- 'noDMARCReportDomain' processed only the first reported email address - now all addresses are processed


added:

- The analyzer as well as the work with IP-addresses dialog are now showing ASN information for IP-addresses (ASN, RIP, Mask).
  These information is queried from routeviews.org (University of Oregon Route Views Project) using DNS.



2020-07-26
fixed in assp 2.6.4 *SPAM-Evaporator* build 20208:

- the remember function for the "and copy the file to correctednotspam folder" setting in the file editor dialog was not working



2020-07-08
fixed in assp 2.6.4 *SPAM-Evaporator* build 20190:

- the resend function using the GUI was not working if 'AddIntendedForHeader' was wrong set in the configuration or the X-Assp-Intended-For or X-Assp-Envelope-From was missing for any other reason


2020-06-30
fixed in assp 2.6.4 *SPAM-Evaporator* build 20182:

- build 20161 and 20181 forced an authentication to the relayhost even no authenication credentials were defined. This caused an error:

  error: authentication failed (535 Authentication failed. Restarting authentication process.) - try to continue unauthenticated


2020-06-29
fixed in assp 2.6.4 *SPAM-Evaporator* build 20181:

- resend a mail using the GUI was no longer working in build 20161


2020-06-09
fixed in assp 2.6.4 *SPAM-Evaporator* build 20161:

- header lines with an email address in the comment part like "<user@domain.org>" were wrong parsed. This was leading in to wrong SPF,DMARC and FROM checks

- the "INBOUND" keyword in SMTP listener definitions was not accepted by the GUI


added:

- the hidden configuration parameter 'AUTHrelayTable' can be used to map different authentication values to different relayHosts(s)

our $AUTHrelayTable = {};                # HASH to lookup authentication credentials for different relayHost(s)
                                         # if this HASH is empty or no host matches relayAuthUser and relayAuthPass are used
                                         # if any of relayAuthUser and relayAuthPass is not defined, no authentication will be done to the relayHost
                                         # example:
                                         # $AUTHrelayTable = {relayHost1:relayPort1 => [relayuser1,relaypass1],
                                         #                    relayHost2:relayPort2 => [relayuser2,relaypass2],
                                         #                    relayHost3:relayPort3 => [relayuser3,relaypass3]}



2020-03-25
fixed in assp 2.6.4 *SPAM-Evaporator* build 20085:

- the analyzer has not shown the right RWL-state, if the RWL result was provided by the RWL-cache

- REQUIRETLS is now defined by IANA in the RFC8689 - SMTP error reply codes got a small change


2020-02-25
fixed in assp 2.6.4 *SPAM-Evaporator* build 20056:

- mails blocked by the IP-blocking feature or PBextreme were some times stored without the mail body and for this reason forwarded incompletely



2020-02-13
fixed in assp 2.6.4 *SPAM-Evaporator* build 20044:

- the default value of 'ConnectionLog' is changed from 'noLog' (0) to 'standard' (1)

- the SMTP-worker was dieing, if an invald 'ARC-Authentication-Results' header was defined

- in some cases the warning 'warning: to remove an IP-address or IP-address-range from a defined IP-address-range, you need to install the modules Net::IP and NetAddr::IP::Lite'
  was written to maillog.txt, even both modules were installed



2020-02-06
fixed in assp 2.6.4 *SPAM-Evaporator* build 20037:

- MIME encoded email addresses in header fields are now decoded before any email address check is done.



2020-01-30
fixed in assp 2.6.4 *SPAM-Evaporator* build 20030:

- the GUI option 'ssl_version' did not allow to configure TLSv1_3


2020-01-02
fixed in assp 2.6.4 *SPAM-Evaporator* build 20002:

- public release


2019-12-16
fixed in assp 2.6.4 *SPAM-Evaporator* build 19350:

- ASSP_ARC.pm 2.09 is released - it prevents Storable from croaking about invalid data structures, which may caused unexpected worker exceptions

changed:

- minor logging changes related to 'AUTHLogUser' and 'AUTHLogPWD'



2019-12-07
fixed in assp 2.6.4 *SPAM-Evaporator* build 19341:

- remote support was not working, if connections limits were configured

- If a DKIM-identiy tag (i=.....) was calculated for a DKIM-signature, this DKIM-identity was cached and unexpected added to all later calculated DKIM-signatures until assp was restarted,
  which made this DKIM-signatures invalid.


changed:

- 'AddSubjectHeader' is now restricted (added only) to incoming mails




2019-11-20
fixed in assp 2.6.4 *SPAM-Evaporator* build 19324:

- Depending on their content and the used perl version, it was possible that since build 19309 several regular expressions were not working like expected in any worker thread.
  Most times the regexes were working case sensitive (NOT case sensitive is the default) on affected systems.

- Several parameters in the ASSP-MIB.txt file contained an underscore (_) , which is not allowed and may prevented the snmp daemon from loading the MIB file.
  A new ASSP-MIB.txt is provided and both MIB-creator scripts

  lib/SNMPmakeMIB.p_
  lib/SNMPmakeMRTG.p_

  are updated to create syntactical right MIB files



2019-11-12
fixed in assp 2.6.4 *SPAM-Evaporator* build 19316:


- if a mail matched the 'contentOnlyRe', the 'HELO' and PenaltyBox checks were not skipped



changed:

- enhanced perl version check and reporting

- it is now possible to use the regex-eval-code function in regular expressions without any restriction
  how ever - it is highly recommended to NOT enable the required hidden feature ('AllowCodeInRegex'). If regex files are not protected from unauthorized write access,
  assp/perl can be forced to execute any code at runtime!

- if a mail matched the 'contentOnlyRe', the connected IP was undocumented treated like an ISP-IP - this is no longer the case

- code indent correction


added:

- a new hidden parameter is added to control the SPF PASS action
  !!! ATTENTION !!! : until now this action was hardcoded corresponding to a setting of 5 - now the default is 0

our $SPFpassAction = 0; # (0..7) if SPF  pass: bit-0 = set rwlok to 1 (medium trust status), bit-1 = skip penaltybox-check, bit-2 = set IP-score to zero - default is 0 (no bits set)



2019-11-05
fixed in assp 2.6.4 *SPAM-Evaporator* build 19309:

- IP-blocking was unexpected skipped, if PBextreme-blocking was not configured

- even if 'AllowCodeInRegex' was enabled, regular expressions with evaluation code were rejected by the regex-precheck
  known issue for 'AllowCodeInRegex': perl/assp variables used in regular expressions evaluation code are read and interpolated at the regex configuration time
                                      if such a variable is changed after configuration, this change is ignored inside the regular expression
  how ever - it is highly recommended to NOT enable this hidden feature ('AllowCodeInRegex'). If regex files are not protected from unauthorized write access,
  assp/perl can be forced to execute any code at runtime!



2019-10-30
fixed in assp 2.6.4 *SPAM-Evaporator* build 19303:

- if 'FileLogScan' or 'ClamAVLogScan' was configured to scan collected files and 'SpamVirusLog' was set to 'no collect' the post virus scan has not removed the infected file,
  instead it was copied to the quarantine folder


changed:

- noModuleAutoUpdate' is now working on nix systems for non root users, if CPAN is configured to use sudo (without password prompt) for the running assp user



2019-10-25
fixed in assp 2.6.4 *SPAM-Evaporator* build 19298:

changed:

- related to the changes in build 19297 assp will try to consolidate the tables, if a table with capital letters in the table name is found
  where consolidate means: 
  - assp tries to rename the table to lower case
  - if the rename failes because the lower case table already exists, data is copied from upper case to lower case tables and the table with capital letters is dropped
  
  

2019-10-24
fixed in assp 2.6.4 *SPAM-Evaporator* build 19297:

- If MySQL 5.7 (or higher) or MariaDB was used as database backend at a case sensitive filesystem and the mysql configuration switch 'lower_case_table_names = 1'
  ( https://dev.mysql.com/doc/refman/5.7/en/identifier-case-sensitivity.html ) was not defined in the MySQL Server configuration file, it was possible that assp database imports
  and the assp database connection keeping failed.
  
  It was also possible, that several database tables for dynamic PB-caches and lists were mistakenly created twice by assp - one time with all lower case characters and
  one time with some capital letters in the name. From now on assp uses lower case database table names in every case.
  
  the following database tables may be affected by this issue, depending on your assp configuration: 
          adminusers, adminusersright, backdns, batvtag, dkimcache, mxacache, pbblack, pbtrap, pbwhite, ptrcache, rblcache, rwlcache, sbcache, spfcache, uriblcache

  *****************************************
  How to fix twice created database tables?
  *****************************************
  
  Be prepared. Be not in a hurry. Read all the next lines!
  In doubt contact your database administrator for assistance!
  
  **** ATTENTION ****
  While working with database tables, DO NOT change the assp root password - this may lead in to data lost and/or startup crashes!
  
  option 1: stop assp - drop tables which contains capital characters: DROP TABLE 'UpperCaseTableName' - start assp
  
  option 2: stop assp - rename tables which contains capital characters: ALTER TABLE 'UpperCaseTableName' RENAME TO 'lowercasetablename' - start assp
  
  option 3: use the assp buildin functions
            - make sure your assp is doing regular database backups (importDBDir backupDBDir backupDBInterval copyDBToOrgLoc)
            - set 'fillUpImportDBDir' to L and apply - this will copy the last backup of all tables to the 'importDBDir'
            - stop assp
            - possibly remove the import files for already lower case table names from the 'importDBDir'
            - DROP all tables from the database which contains upper case letters
            - start assp - assp will create all missing tables and will import all data for all tables from the 'importDBDir' at startup
  
  option 4: use the mysql client, the workbench, the MySQL-Tools or any other DB management tool (like DBeaver)
            - stop assp - use the management tool of your choice to backup, rename, drop, recover the required tables and there data - start assp

  Notice: it is save to drop all tables which contains only dynamic PB-cache data - these table will be filled up by assp again quickly
          if the tables 'adminusers' and/or 'adminusersright' contain capital letters and valid data, handle them with care to prevent data lost !!!
          if you upgrade MySQL or MariaDB consider to set the configuration switch 'lower_case_table_names = 1' to force the DB-engine to use lower case table names
              !!! BUT !!! - before you set the MySQL configuration switch 'lower_case_table_names = 1' make sure all existing table names are lower case unique!

  *************************************************************************************
  What happens if you consider to do nothing - or an automatic assp.pl upgrade is done?
  *************************************************************************************
  
  Don't worry - if assp detects a case mismatch for a database table at startup - it will try to rename the table to lower case. If this is not possible for any reason, the lower case
  table will be checked or created and used. ASSP will write a startup warning to the maillog.txt for this event. The dynamic caches and lists will be filled up quickly. ASSP will do some more DNS-queries
  as normal for a short time. You should consider to remove the uppercase database tables, if you find some time later.
  If there are admin users defined, it may be possible that they are unable to login to the GUI. In this case, follow the instructions above (How to fix....) 
  to rename both tables (adminusers, adminusersright). The assp root account is not affected.




2019-10-21
fixed in assp 2.6.4 *SPAM-Evaporator* build 19294:

- if perl was started using the -w flag, multiple warnings about the usage of uninitialized variables were shown

- an error: 'error: DKIM-cfg - no configuration left for any domain' was logged, if a valid DKIM-configuration was stored, but 'genDKIM' was not enabled and 'genARC' was not enabled


changed:

- the updated buildin rebuildspamdb.pm version 7.51 got some small speed improvements

- ASSP_RSS.pm is updated to version 1.11 - unneeded code is removed

- assp.pl code optimization is started based on the perl development version 5.31.5, to become compatible to the next year upcomming perl production version 5.32
  target is to be able to switch to perl 5.32 as soon as it is released, without any need to wait

- if 'enableATA' is enabled in ASSP_AFC, forwarded (copied) spam mails now gets the 'ATAHeaderTag' included, if they contain attachments or includes
  this prevents mistakenly quarantine released mailicious mails from bypassing the ATA



2019-10-11
fixed in assp 2.6.4 *SPAM-Evaporator* build 19284:


- disabled 'ScoreForeignCountries' was ignored in some cases

- mails were unexpected blocked, if 'disableAUTH' was enabled and the sender used the AUTH=<> extension in the MAIL FROM: SMTP command (RFC4954)



changed:

- the ASSP_VirusTotal_API version is shown in the module statistic - load error are recorded

- ASSP_OCR.pm 2.23 with improved error handling and logging




2019-09-27
fixed in assp 2.6.4 *SPAM-Evaporator* build 19270:

- the whitelist state was some times wrong shown in the 'work with lists' GUI panel


changed:

- the 'work with lists' GUI panel got several design and functional improvements (details are shown in the GUI)

- ASSP_AFC.pm version 5.15 is able to detect encrypted OLE-objects - such OLE-files are classified as 'NOT harmless'

-  the OpenSSL and OpenSSL-Library minimum version requirements are changed to
  OpenSSL: 1.0.0t
  OpenSSL-Library: 1.1.0h



2019-09-03
fixed in assp 2.6.4 *SPAM-Evaporator* build 19246:


- improved parsing of the block-report RSBM_..... recipient address (right resend link), in case the address was converted to lower case by the mail client
  related to: https://sourceforge.net/p/assp/tickets/111/


changed:

- the following deprecated modules are removed from the code:
  Email::Send
  Return::Value

- the following vulnerable module is remove from the primary code
  Email::Address (CVE-2015-7686)

- the recommended version of Email::MIME is changed from 1.936 to 1.946

- the module Email::Address::XS 1.04 is required (automatically installed by the latest Email::MIME) - if not installed Email::Address is used as fallback





2019-08-02
fixed in assp 2.6.4 *SPAM-Evaporator* build 19214:


- If assp was started after a perl version upgrade (eg. as service or daemon), without installing the possibly required perl module Crypt::GOST 1.01 (eg. running the assp module installer again),
  all encypted configuration parameters and files were destroyed.
  ASSP now checks at every startup, that the required Crypt::GOST module is installed. If this module is required but missing, ASSP will end immediatly and will write the following error to STDOUT,
  STDERR and the file moduleLoadErrors.txt.
  
***** ERROR ***** ERROR ***** ERROR *****
***** CAN NOT START ASSP - incompatible encryption engine in use *****
***** ERROR ***** ERROR ***** ERROR *****
ERROR: last time assp was started, the perl module Crypt::GOST was available and used - but now it is not available for any reason (eg. perl upgrade without running the assp module installer script).
Please read the installation instructions (eg. run the ./assp.mod/install/mod_inst.pl script) - or install the missing module Crypt::GOST.
The source of Crypt::GOST is available at ./assp.mod/Crypt-GOST-1.01.src.tar.gz or at https://sourceforge.net/projects/assp/files/ASSP\%20V2\%20multithreading/ASSP\%20V2\%20module\%20installation/Crypt-GOST/Crypt-GOST-1.01.src.tar.gz .
It is mandatory to run the assp module installer script (and to correct all shown errors) before starting assp.
It is highly recommended to run 'cpan-outdated -p|cpanm [-n]' or 'cpan> [notest] upgrade' to bring all perl modules uptodate.
ASSP will not start until the perl module Crypt::GOST is installed, or its reference is removed from ./notes/loaded_perl_modules.txt .
If this reference is removed without installing the Crypt::GOST module, assp will start - but all encryped configuration values and files will become invalid and you'll need to reconfigure them all manually!
***** ERROR ***** ERROR ***** ERROR *****


2019-07-14
fixed in assp 2.6.4 *SPAM-Evaporator* build 19195:

- the 'two-level-tlds' download URL is changed from http://george.surbl.org/two-level-tlds to http://www.surbl.org/tld/two-level-tlds - george.surbl.org is no longer a valid hostname

- the 'three-level-tlds' download URL is changed from http://george.surbl.org/three-level-tlds to http://www.surbl.org/tld/three-level-tlds - george.surbl.org is no longer a valid hostname



2019-07-02
fixed in assp 2.6.4 *SPAM-Evaporator* build 19183:

- if 'genARC' was enabled and foreign Authentication-Results: headers were present in the mail, no ARC-signature was created and an error
  "error: ARC message parsing failed - unexpected token at /usr/local/lib/perl5/site_perl/Mail/DKIM/ARC/Signer.pm line 254 thread ..."
  was written to the maillog.txt



2019-06-24
fixed in assp 2.6.4 *SPAM-Evaporator* build 19175:


changed:

- the analyzer now also ask VirusTotal for results, if this option is enabled in ASSP_AFC

- ASSP_AFC.pm version 5.12 now detects faked RTF documents as executable content (cve-2012-0158, cve-2017-11882 and others)


2019-06-18
fixed in assp 2.6.4 *SPAM-Evaporator* build 19169:

- the file upload feature in the ASSP-filecommander has sometimes destroyed the uploaded files (+ was replaced by space)
  this issue is now finaly fixed
  

changed:

'DKIMpassAction' is moved back to a hidden parameter. The default value is set to 0
  



2019-06-09
fixed in assp 2.6.4 *SPAM-Evaporator* build 19160:

- the bug fix in build 19157 related to the ASSP-filecommander is reverted - it caused problems with files edited in the GUI


2019-06-06
fixed in assp 2.6.4 *SPAM-Evaporator* build 19157:

- the file upload feature in the ASSP-filecommander has sometimes destroyed the uploaded files (+ was replaced by space)

- VirusTotalAPIKey was rejected in every case, because of a validation check bug


added:

'DKIMpassAction' is moved from a hidden to a GUI parameter and default value is changed from 7 to 0

'DKIMpassAction','Special Action if DKIM passes'
 'Special action on message processing, if the DKIM check is passed.
 This value is a bit-mask using bit 0 to 2. So, valid values are in the range from 0 to 7. Default value is 0 - no special action
 Setting a bit to 1, will force the according action. The resulting value is the sum of the decimal values of the bits (1,2,4)
 - bit-0 (1) = set the message flag "rwlok" to 1 ( RWL low trust status, the same like in RWLServiceProvider )
 - bit-1 (2) = skip the penaltybox-check ( same like the IP is listed in noPB )
 - bit-2 (4) = set the IP-score (not the message scores!) to zero - IP penalty scores possibly counted after the DKIM-check, keep active.'




2019-05-31
fixed in assp 2.6.4 *SPAM-Evaporator* build 19151:

- 'fillUpImportDBDir' was not working on some systems

- a good rule '.*' in UserAttach was ignored


added:

- queries for viruses and bad URL's to www.virustotal.com are now supported
  virus checks require ASSP_AFC.pm (version 5.10)

lib/ASSP_VirusTotal_API.pm (version 1.01) and the changed ASSP_AFC.pm (version 5.10) and

'VirusTotalAPIKey','The Privat API-Key for VirusTotal'
 'To query www.VirusTotal.com for URIs and/or viruses (ASSP_AFC.pm), a valid API-Key is required. An API-Key is provided by VirusTotal for free, after your registration at www.virustotal.com.
 Such a free API-Key is limited to four queries at VirusTotal per minute. API-Keys for a higher query volume are also provided by VirusTotal.
 Systems that are part of the ASSP-Global-PenalyBox network can leave this value empty. They are getting an API-Key with a much higher query volume from the GPB-Server automatically,
 without any additionally costs. This API-Key is not shown here!'

'ASSP_AFCDoVirusTotalVirusScan','Enable VirusTotal Virus Scan'
'If a VirusTotalAPIKey is provided and this option is enabled, all MIME-parts will be (in addition to ClamAV and/or FileScan) checked by www.virustotal.com.' 


- DBD::MariaDB is now supported


changed:

'enhancedOriginIPDetect','Do an Enhanced Origin IP Address Detection in the Mail Header'
  Local and private IP's, IP's assigned by IANA to the Shared Address Space (100.64.0.0/10 RFC6598) and IP's listed in ispip, acceptAllMail, whiteListedIPs, noProcessingIPs, noDelay and noPB
  will be ignored.

'RBLServiceProvider','RBL Service Providers*'
references to  combined.njabl.org are removed from the GUI

'URIBLServiceProvider','URIBL Service Providers*'
...
 If VirusTotalAPIKey is configured, assp is able to query URIs on www.virustotal.com . The API answers are in the range 127.0.0.2-127.0.0.253 (or none for OK), where the last digits represents HITS + 1.
 Queries to VirusTotal are using HTTPS connections (https://www.virustotal.com/...) instead of DNS!
 example:
 virustotal=>127.0.0.2=>1 # one hit
 virustotal=>127.0.0.3=>0.5 # two hits
 virustotal=>127.0.0.4=>0.33 # three hits
 virustotal=>127.0.0.*=>0.25 # more than three hits'







2019-04-25
fixed in assp 2.6.4 *SPAM-Evaporator* build 19115:

- the post virusscan for the stored corpus files, scored for the already finished mail - this was confusing for some users and id removed

- HTML-comments are now removed from resend request emails, because there content may has affected the resend processing 



2019-03-27
fixed in assp 2.6.4 *SPAM-Evaporator* build 19086:

- The ClamAV-engine now uses the modern INSTREAM clamav-API. It uses less system resources and is faster than the "old" STREAM-API.


changed:

- The default value for 'ClamAVtimeout' is changed to 30 seconds.



2019-03-26
fixed in assp 2.6.4 *SPAM-Evaporator* build 19085:

- Several domains provide their SPF-record (and possibly other DNS-records) as wildcard records (for each possible subdomain).
  This caused the DKIM-preCheck to detect a (possible) provided DKIM-DNS-configuration, because it got a TXT record (the wildcard-record) for _domainkey.domain.tld and/or _adsp._domainkey.domain.tld.
  Not DKIM related DNS TXT answers are now ignored by the DKIM-preCheck to prevent false positives.



2019-03-25
fixed in assp 2.6.4 *SPAM-Evaporator* build 19084:

- the resend from block report using the right button failed, if the subject of the mail contained 'x' followed by two digits (eg: x30)

- using the unix socket for the ClamAV communication failed on some systems

- assp has thrown an error if the ClamAV, configuration was anyway invalid or not working, but UseAvClamd was disabled

- the rebuildspamdb task crashed, if the HMMdb contained only one record

- ASSP_AFC.pm version 5.04 is released

  ASSP_AFC.pm is now able to tell a local mail server or andvanced thread analyzer, if the attached files may need some further investigation or analysis
  This is done by adding a special (hiddenly configurable) MIME header tag.

# advanced thread analyzing or deep thread inspection for incoming mails
$ASSP_AFC::enableATA = 0;         # 1- check ATA if an attachment failed, 2- check if any attachment is found, 3- check every mail
$ASSP_AFC::ATAHeaderTag = "X-ASSP-Require-ATA: YES; RESENDLINK;SHOWMAIL;SHOWLOG\r\n"; # the literal RESENDLINK will be replaced by a mailto resendlink, which may be shown by an ATA report mail
                                                                                # SHOWMAIL offers the link to open the file in the assp file editor
                                                                                # SHOWLOG offers the link to show the log for the mail in maillogtail (an optional trailing number defines the days in the past e.g. SHOWLOG2 for example - two days is default and used if no number is given)
                                                                                # every link is preceeded by \r\n\t 





2019-01-19
fixed in assp 2.6.4 *SPAM-Evaporator* build 19019:

- the analyzer got changes to fully support ASSP_AFC 5.02 

changed:

- ASSP_AFC 5.02 is released - it contains fixes and extensions for 'ASSP_AFCKnownGoodEXE','Well Known Good Executable Files'

[ASSP_AFCKnownGoodEXE,'Well Known Good Executable Files'
 'Put the SHA256_HEX hash of all well known good executables in to this file (one per line). If the SHA256_HEX hash (not case sensitive) of an attachment or a part of a compressed attachment
 (e.g. exe, *.bin MS-Macro or OLE) is equal to a line in this file, the attachment passes the attachment check for all mails (regardless its extension and the settings in UserAttach).
 The same applies to the following ojects in a PDF file: Certificate, Signature, JavaScript . If the SHA256_HEX hash of any of these PDF objects matches, the PDF will pass the attachment check.
 Comments are allowed after the hash and at the begin of a line (recommended).
 If configured, the analyzer and the maillog.txt will show the SHA256_HEX hash and the optional defined comment for all detected executables and PDF objects.
 For security reasons, virus scanning is not skipped.
 <b>Notice:</b> this feature is mainly created for executable files, but it will work for every attachment and every part of a compressed attachment.
 For example - this can be usefull, if clients regular sending or receiving documents or excel sheets, which contains every time the same MS-Macro/MS-OLE (e.g. executable).
 In this case, decompress the doc[xm] and calculate the SHA256_HEX hash for the vbaProject.bin or the vbaProjectSignature.bin file and register the hash here.
 examples:
 
 # sales documents
 a704ebf55efa5bb8079bb2ea1de54bfd5e9a0f7ed3a38867759b81bfc7b2cc9c # sales price_list.pdf - contains well known good Java-Script
 96c4e6976d16b424ff02d7ef3fdabf41262d3ffc6a191431dc77176a814c1256 # sales sales_report.pdf - contains known Certificate
 08d5518ef129ba1a992f5eb5c25e497cf886556710ffebe7cfb6aedf9d5727c9 # VBA Macro signature vbaProjectSignature.bin in sales info.docm
 
 In addition to the SHA256_HEX hash, you can define at which compression level the hash should be valid. Compression levels are comma separated numerical values or ranges
 - like 0,1,2 or 0-2 or 0...8 or 0-2,4...6 or 1 .
 The compression level zero is the not decompressed attachment itself. To include all compression levels, define a single asterix * or no level definition.
 examples:
 
 # sales documents
 a704ebf55efa5bb8079bb2ea1de54bfd5e9a0f7ed3a38867759b81bfc7b2cc9c 0,1 # sales price_list.pdf - contains well known good Java-Script - valid at zip level 0 and 1
 96c4e6976d16b424ff02d7ef3fdabf41262d3ffc6a191431dc77176a814c1256 *   # sales sales_report.pdf - contains known Certificate - valid at any zip level
 08d5518ef129ba1a992f5eb5c25e497cf886556710ffebe7cfb6aedf9d5727c9 1   # VBA Macro signature vbaProjectSignature.bin in sales info.docm - only valid in the .docm itself (which is a zip) - .docm in a zip is not valid
 08d5518ef129ba1a992f5eb5c25e497cf886556710ffebe7cfb6aedf9d5727c9 0   # VBA Macro signature vbaProjectSignature.bin in sales info.docm - this will not work, because a .docm is a compressed file
 
 To show the SHA256_HEX value for a file at the command line, execute :>shasum -a 256 -b the_file_name
 To show the SHA256_HEX values for all relevant PDF-objects in a PDF file, change in to the assp folder and execute :>perl getpdfsha.pl the_PDF_file_name .
 You may also compose and send a mail with the files in question attached to the analyze email-interface - EmailAnalyze . 
 The log output of the analyzer will show all SHA256_HEX hashes (if AttachmentLog is enabled).
 Notice: different PDF creator applications may store the same PDF-object (Cert, Sig, JS) in different ways, which will result in different SHA256_HEX hashes for the same PDF-object!
 If this happens, you need to calculate the SHA256_HEX hash for each different occurence of the PDF-object.'






2019-01-15
fixed in assp 2.6.4 *SPAM-Evaporator* build 19015:

added:

- ASSP_AFC 5.01 is released - it includes a new extension

 'ASSP_AFCKnownGoodEXE','Well Known Good Executable Files'
 'Put the SHA256_HEX hash of all well known good executables in to this file (one per line). If the SHA256_HEX hash (not case sensitive) of an attachment or a part of a compressed attachment 
 (e.g. exe, *.bin MS-Macro or OLE) is equal to a line in this file, the attachment passes the attachment check for all mails (regardless its extension and the settings in UserAttach).
 Comments are allowed after the hash and at the begin of a line.
 If configured, the analyzer and the maillog.txt will show the SHA256_HEX hash and the optional defined comment for all detected executables.
 For security reasons, virus scanning is not skipped.
 Notice: this feature is mainly created for executable files, but it will work for every attachment and every part of a compressed attachment.
 For example - this can be usefull, if clients regular sending or receiving documents or excel sheets, which contains every time the same MS-Macro/MS-OLE (e.g. executable).
 In this case, decompress the doc[xm] and calculate the SHA256_HEX hash for the vbaProject.bin or the vbaProjectSignature.bin file and register the hash here.
 examples:
 
 # sales documents
 a704ebf55efa5bb8079bb2ea1de54bfd5e9a0f7ed3a38867759b81bfc7b2cc9c # sales price_list.pdf - contains Java-Script
 08d5518ef129ba1a992f5eb5c25e497cf886556710ffebe7cfb6aedf9d5727c9 # VBA Macrco vbaProject.bin in sales info.docm
 
 To show the SHA256_HEX value for a file at the command line, execute :>shasum -a 256 the_file_name'


changed:

- the default value for 'DoNoFromSelect' is changed from 63 to 59
  option 4 - multiple from: addresses or from: header tags found (potential 2x score if option 2 is also enabled) - caused too many false positives



---------------------------------------------------------------------------------------------------------------------------------------------

2019-01-07
fixed in assp 2.6.2 *Fortress* build 19007:

- no functional changes

- this release is published for public as 2.6.1 build 19007



2018-12-31
fixed in assp 2.6.2 *Fortress* build 18365:

- specific unicode regular expressions like \p{Yi} and others - were not working for the MIME header under certain conditions (spam bomb definitions were not affected by this issue)

- improved domain name parsing - the length restiction (63 bytes) for each label is now checked

- assp_pop3.pl version 1.21 is released
  - the SSL mode workaround for old Net::POP versions is removed - at least version 3.07 of Net::POP3 is now required
  - in some exceptional cases it was possible, that an email was retrieved and delivered multiple times


  

2018-12-27
fixed in assp 2.6.2 *Fortress* build 18361:

- assp_pop3.pl version 1.17 is released
  this release contains some small bug fixes and a POP3-connection retry, in case the POP3-server closed the connection unexpected
  

added:
 
- using the command line switch 'checkLinuxENV:=n' or setting $main::checkLinuxENV=n; in 'lib/CorrectASSPcfg.pm', assp will the ulimit settings and the selinux state on nix systems
  in case any settings seems to be too less, warnings or errors are shown at startup

our $checkLinuxENV = 0;   # (0/1/2) check ulimit (1) on nix and selinux (2) on linux systems


changed:

- 'DoNoFromRemovesNPWL' is now moved to the GUI

'DoNoFromRemovesNPWL','DoNoFrom Removes NP, WL Flag','0:disabled|1:whitelisted|2:noprocessing|3:both'
 'If the combination of DoNoFrom , DoNoFromSelect , DoNoFromWL and DoNoFromNP gives more than one hit, the whitelisted and/or the noprocessing flag will be removed from the message.
 For example: if the FROM: and /or SENDER: address fakes a whitelisted and/or noprocessing address or domain.
 Default setting is both.
 The noprocessing by size flag ( npSize ) will be keeped.'




2018-12-17
fixed in assp 2.6.2 *Fortress* build 18351:

- If the daily amount of collected .eml files in one folder exceeded the value of 'MaxFiles', new files were removed by the daily
  file cleanup processing - which caused failing resend requests. New files are now keeped for at least five days, even the file count
  exceeds the value of 'MaxFiles'. Set 'MaxFiles' high enough, to keep files for a longer periode.
  
- ASSP_AFC.pm 4.88 fixes a BUG where a missdetection of MIME-file-types prevented the decompression of zip files

- The file edit dialog got an additionally option, to resend a blocked mail and to copy the blocked spam file to correctednotspam at the same time.



2018-12-05
fixed in assp 2.6.2 *Fortress* build 18339:

- BerkeleyDB engine version 18.1 was detected as too old

added:

- It is now possible to remove the whitelisted and/or the noprocessing flag from a mail, if DoNoFrom gets more than one hit
 Notice: DoNoFromWL and/or DoNoFromNP have to be enabled to be able to remove the flags

our $DoNoFromRemovesNPWL = 0;            # (0/1/2/3) DoNoFrom removes if more than one hit: 
0 - no action, 
1 - whitelisted, 
2 - noprocessing, 
3 - whitelisted and noprocessing 

--- noprocessing by size will be keeped



2018-12-03
fixed in assp 2.6.2 *Fortress* build 18337:

- DoNoFrom detected email addresses in the text part of the header text - like: "do not detect this address user@domain.com but the next one" <other.user@other-domain.org>

- under rare conditions the file name in a blocked mail resend request was wrong parsed, the file was'nt found and the resend failed


added:

- 'DoNoFromSelect','Select Checks for From: and Sender: Header'
 Select which check should be done in DoNoFrom .
 
 1 - from: and sender: header tag are both missing
 2 - different domains found in from: and sender: email addresses
 4 - multiple from: addresses or from: header tags found
 8 - multiple sender: addresses or sender: header tags found
 16 - no or an invalid email address found in from: header tag
 32 - no or an invalid email address found in sender: header tag
 
 Simply form the sum of the numbers in front of the checks you want to select (0...63). Default vaule is 63 (1+2+4+8+16+32) - all checks are selected.'


changed:


- $DoNoFromDomainCHK is removed - use DoNoFromSelect instead




2018-11-24
fixed in assp 2.6.2 *Fortress* build 18328:

- it was possible that bomb.. checks were matching with an empty string result - now only bombSubjectRe can match on an empty string



2018-11-22
fixed in assp 2.6.2 *Fortress* build 18326:

- using the search option in MaillogTail has show expected results

- faster search in MaillogTail

- an entry like "user@domain.com" <user@domain.com> in any header tag was missinterpreted as tow email addresses


added:

our $DoNoFromDomainCHK = 1;              # (0/1) enable the domain check for DoNoFrom (different domains in FROM: and SENDER: header are a fault)

Set the value to undef or zero to disabe this check.




2018-11-13
fixed in assp 2.6.2 *Fortress* build 18317:


- bad MIME encoded multiline headers were some times wrong decoded
  (in some mails, header lines were broken in to multiple MIME encodings at any byte instead at a character)

- the "secured browser sandbox" introduced in build 18316 was not working in MS Internet Explorer
  For security reasons images are never shown in MS Internet Explorer, when "show email in browser sandbox" is requested.



2018-11-12
fixed in assp 2.6.2 *Fortress* build 18316:


changed:

The .eml file editor dialog got some new options

- the action pulldown menu got an option to force the resend of the email including ALL attachments 'copy file to resendmail and force attachments' (shown in red color !) 

- an action button "show email in browser sandbox" is added
 Using the left mouse button at "show email in browser sandbox" will show the email in a secured browser sandbox "https://en.wikipedia.org/wiki/Content_Security_Policy" (Content Security Policy),
 using the right mouse button, images will be show in addition. Showing images can be a risk, if they contain malicious code!

- If the email contains attachments or includes, a hint is given and the attachments are listed as links. Clicking on such a link will download the attachment to the local machine.
  This may be used to check attachments for malicious content before a resend is requested. 




2018-11-09
fixed in assp 2.6.2 *Fortress* build 18313:

- reduced memory footprint for GUI request handling

- a SMTP worker was in rare cases dieing, because syswrite was unable to process wrong encoded emails or attachments


changed:

- 'DoNoFrom' now also checks for multiple FROM: or SENDER: email addresses in a single header tag

- the default value for 'DoNoFromWL' and 'DoNoFromNP' is change to 1 (checked)

- the internal attachment blocking feature (without using the ASSP_AFC plugin) now allows to detect extended file extensions, like .... .tar.gz or .... .tar.gz.aes 
  (ASSP_AFC was and is able to handle those file extensions)


2018-10-31
fixed in assp 2.6.2 *Fortress* build 18304:

- after upgrading the perl module Win32::Daemon to version 20181025 assp was no longer starting as a windoes service
  this assp version contains a workaround for the buggy Win32::Daemon module
  
- a small memory leak is solved in unicode processing for perl 5.22 to 5.28

- the detection of incoming DMARC-reports is improved

- the not RFC conform DMARC-reports from "Amazon SES" are now correctly detected

- perl module load errors in ASSP_AFC were not shown in the file moduleLoadErrors.txt - ASSP_AFC.pm is updated to version 4.87 to get this fix working

- some of the X-ASSP-... headers were some times too long (RFC822, RFC 1522)


changed:

- backscatter checks are skipped for regular incoming mails (not matching redRe) for local postmaster@ and webmaster@ addresses, even these addresses are listed in BounceSenders
  



2018-10-22
fixed in assp 2.6.2 *Fortress* build 18295:

- The DMARC check now follows the RFC7489 for the blocking rules. The DMARC-check is OK, when SPF or DKIM passes there check.

- If an email was blocked by the SPF-check, no DMARC-report was generated for this email.

- The DKIM-precheck is improved to detect, if a domain supports DKIM or not.



added:

- ASSP supports now the SMTP extension 'Require-TLS - REQUIRETLS' https://tools.ietf.org/html/draft-ietf-uta-smtp-require-tls-04
  This is just a Draft in version 04 - for this reason, the feature is still experimental.
  The following hidden configuration parameters are used by this feature.

  our $enableREQUIRETLS = 0;          # (0/1) enable testing of the REQUIRETLS implementation
  our $provideREQUIRETLS = 0;         # (0/1) include REQUIRETLS in to the EHLO reply if not already provided
  our $forceREQUIRETLS = 0;           # (0/1) include REQUIRETLS in to the MAIL FROM: command if not provided by the MTA


changed:

- If DMARC is enabled and a NDR is received for a sent DMARC-report for any reason and 'noDMARCReportDomain' is configured using the 'file:...' option,
  the foreign report recipient address and/or the report domain are automatically added to 'noDMARCReportDomain'.

- If a domain provides an explicite _adsp policy with the value 'unknown', the domain is no longer added to the DKIMCache and does no longer require to sign all mail using DKIM,
  if 'DKIMCacheStrict' is not set. 
  The behavior is not changed for the case where the _adsp policy with the value 'unknown' is not explicite defined and falls per default to 'unknown' or 'DKIMCacheStrict' is set.

- The mail analyzer now shows results with more details for the DMARC-check.

- The maillog.txt file list in the 'MaillogTail view' is now shown permanent (unless closed) and contains four columns to provide more files to be shown.



2018-10-15
fixed in assp 2.6.2 *Fortress* build 18288:

- the included rebuildspamdb.pm inreases the rebuild performance by 10 to 20%

- the correction of the spamDB and HMMdb in case of reported spam or ham was too weak


changed:

- a new ASSP-MIB file is available and required for this version if SNMP is used

- the default value for 'backupDBInterval','backup database Interval' is changed from 2 to 12 hours

- the description for 'SNMP' is changed

...
The following OIDs (relative to the SNMPBaseOID) are available for SNMP-queries. The configuration values are changeable via snmp. The published file mib/ASSP-MIB, 
which contains all possible OID's, could be used in SNMP browsers to get a human readable view of the OID's (copy it to the net-snmp MIB file location - eg: [C:]/usr/share/snmp/mibs 
and the MIB location of your SNMP browser). Please keep in mind, that an extensive usage of SNMP queries will slow down assp.
Because the OID numbers can change in different assp versions, it is recommended to query the OID's by its consistent name (not by its number). This requires the usage of the assp version 
compatible mib/ASSP-MIB file!
If you want to query or set any of the following configuration parameters: LocalAddresses_Flat, LocalAddresses_Flat_Domains, noBayesian_local, Bayesian_localOnly, SSL_version, SSL_cipher_list - 
remove all underscores from the config name to build the OID-name, because underscores ar not allowed in SNMP queries. The MIB file already contains the corrected names.
If you get unexpected SNMP-query results or you've lost the version compatible MIB file, rename the perl scripts lib/SNMPmakeMIB.p_ and lib/SNMPmakeMRTG.p_ to *.pl and restart assp. 
This will create the mib/ASSP-MIB and mib/assp-mrtg.cfg files, based on your installation and configuration. It is recommended to rename both scripts back, after the new MIB files are created.
NOTICE: If you install or uninstall any plugin or you enable or disable the configuration synchronization and you use such a custom MIB file, the mib/ASSP-MIB file needs to be recreated 
to implement the new OID's and (at least) to correct the new OID order!
To prevent permantly copying the changed mib/ASSP-MIB file to your net-snmp daemons MIB-folder - (e.g.) create a link there to the mib/ASSP-MIB file.
...

- the check (and ignore) whitelisted and redlisted mails in the rebuildspamdb task is now disabled per default
  to return to the old behavior set the hidden parameters 'DoRBWhite' and/or 'DoRBRed' to 1.



added:

'spfValencePB' is no longer scored in case DMARC failed - instead 'dmarcValencePB' is used
'dmarcValencePB','DMARC Failed, default=10 +'



2018-10-08
fixed in assp 2.6.2 *Fortress* build 18281:

- the analyzer has wrong shown nonascii characters in the analyzed file name  

- depending on the modes set for 'ValidateSPF' and 'DoDKIM', the DMARC-check was unexpected using monitoring-mode instead of scoring-mode




2018-10-05
fixed in assp 2.6.2 *Fortress* build 18278:

- the (SPF) statistic counter was not working for failed DMARC checks

changed:

- 'DoNoSpoofing4ReplyTo','Do NoSpoofing for Reply-To:' now also processes 'Return-Path:' and 'Disposition-Notification-To:' addresses.

- If a malformed address is found in any of the following header tags, 
  from, sender, reply-to, errors-to, returnreceipt, return-receipt-to, return-path, disposition-notification-to
  the mail and IP gets a score of'nofromValencePB' for each found malformed address and if 'DoDomainCheck' is enabled, this check failes/scores for each found malformed address.
  





2018-10-04
fixed in assp 2.6.2 *Fortress* build 18277:

- the DMARC check ignored the SPF alignment, if 'DoSPFinHeader' was not enabled



added:

notice: the default behavior of assp is changed for whitelisted and noprocessing envelope sender addresses, domains and IP's!

  It was often the case, that mails from known good external senders were blocked, because they sent mails to a list of envelope recipients - but over the time, some of the recipient were no longer valid.
  ASSP detected this 'invalid recipient' attempt and the known good mail was blocked by the recipient check or the mail/IP got a high penalty and was blocked by the penalty-box.
  The hidden configuration parameter 'ignoreInvalidAddressNPWL' is used to ignore the defined 'invalid recipient action', if a known good sender uses unknown envelope recipents in a sequence
  of multiple envelope recipents. The mail is only blocked, if no valid envelope recipient is left over at the DATA command. If an unknown envelope recipient is used, the sender gets no penalty score,
  but the invalid 'RCPT TO:' command is replied with the permanent error '550 5.1.1 User <xxxxx> unknown'. The connection is not dropped in this case. Such an 'invalid recipient' attempt will also
  not be counted for 'MaxErros'.
  
  consequence: the mail is delivered to all left over valid envelope recipients and the sender will be informed about each invalid recipient (if NDR is supported by the sending server)

  $ignoreInvalidAddressNPWL = 3;       # (0/1/2/3) ignore invalid envelope recipients for whitelisted (2) or noprocessing (1) or both (3) senders and IP's (no score, no connection drop, no error count)

  Until now, the default action of assp was like 'ignoreInvalidAddressNPWL = 0' - this is now changed to 'ignoreInvalidAddressNPWL = 3'
  
  To change back to the old behavior or to change the default behavior, you have two options:
  
  1. start assp with the commandline switch --ignoreInvalidAddressNPWL:=X
  
  2. add the line below to the sub set in 'lib/CorrectASSPcfg.pm'.
  
  $main::ignoreInvalidAddressNPWL = X;
  
  In both cases, X is the configuration value of your choice (0...2).
  


2018-09-28
fixed in assp 2.6.2 *Fortress* build 18271:

This release is published for public as 2.6.1 build 18271.

- The Spamhaus DROP List download URL was changed. 
  ASSP now consoldates the following file in to 'droplist'.
  
  http://www.spamhaus.org/drop/drop.txt
  http://www.spamhaus.org/drop/edrop.txt
  http://www.spamhaus.org/drop/dropv6.txt



2018-09-15
fixed in assp 2.6.2 *Fortress* build 18258:

- File names with nonascii characters were some times wrong encoded written to the maillog.txt on perl 5.28 (only perl 5.28). This caused several problems: blockreports missing the resendlinks,
  maillog tail view missed file links, the GUI was unable to open the files ...
  ASSP_AFC.pm 4.86 and ASSP_ARC.pm 2.07 fixes the same issue.

- Depending on the used perl version and the used OS, it was possible that the loglines for different workers in maillog.txt had some times a wrong timeline order.
  This was caused by different microseconds (OS) counts in different workers. ASSP now keeps the logline order.

- If 'sendNoopInfo' was enabled and 'DoTLS' was set to 'doTLS', the STARTTLS command was some times failing. A hint is added to both GUI descriptions to not enable 'sendNoopInfo' if 'DoTLS' is enabled.
  If assp detects both features enabled at startup, 'sendNoopInfo' will be switched to OFF (silent).
  
- If the 'file:....' option was defined for any of the following configuration parameters

  MSGIDsigAddresses
  onlySpoofingCheckIP
  onlySpoofingCheckDomain
  subjectFrequencyOnly
  LocalFrequencyOnly
  onlyAUTHHeloRe
  BlockResendLinkOnly
  Bayesian_localOnly

  and the defined file was empty, the related feature was not working correctly (skipped).
  

changed:

- If a config file or included file of any configuration parameter was used by or included in to the Groups feature (by accident or stupid settings), the
  configuration was destroyed. ASSP will now check such settings and ignores the configuration files (and included files) in such a case.
  A hint related to this issue is added to the Groups feature GUI description.
  In case of such wrong settings, an error messages will be written to maillog.txt and the healthy state of assp will be set to yellow and recommendations will
  be shown in the assp GUI status page. 

- The shutdown sequence is now faster. Both high workers are now terminated first and SMTP connections that have not reached the SMTP-DATA command are now terminated instead of
  processing them.

- If 'ForceNoValidLocalSender' is enabled, the local address spoofing check is now done before the local sender address validation.
  


2018-09-04
fixed in assp 2.6.2 *Fortress* build 18247:

- the "X-ASSP-KEEP" (build 18246) header was sent to the wrong peer




2018-09-03
fixed in assp 2.6.2 *Fortress* build 18246:

- the hintbox was not working in the MaillogTail view since build 18214

- If a large amount of attachments was processed by ASSP_AFC or any other level 2 plugin, it was possible that the MTA was running in its own SMTP-timeout, because
  no data were sent by assp for a too long time. This assp V2 build and the new ASSP_AFC.pm version 4.85 will prevent this issue by sending an "X-ASSP-KEEP" header to the MTA,
  if the assp smtpIdleTimeout will be reached within the next 15 seconds.
  Keep in mind, that the smtptimeout value of your MTA has to be larger than the value defined in 'smtpIdleTimeout'!



2018-08-28
fixed in assp 2.6.2 *Fortress* build 18240:


added:

- DoNoSpoofing4From is now splitted in to DoNoSpoofing4From and DoNoSpoofing4ReplyTo

'DoNoSpoofing4From','Do NoSpoofing for from:'
  'Do the NoSpoofing check also for header 'from:', 'sender:' addresses.'


'DoNoSpoofing4ReplyTo','Do NoSpoofing for Reply-To:'
  'Do the NoSpoofing check also for header 'reply-to:' and 'errors-to:' addresses.'





2018-08-02
fixed in assp 2.6.2 *Fortress* build 18214:

- GUI links to external webpages were not opened in a new browser window

- ASSP_AFC.pm version 4.84 is released
  If a text attachment was found which had no .txt extension (eg. no extenson, .csv, .json) and the found extension was not allowed (the attachment was blocked or replaced),
  the blocking reason was missing in the replaced attachment.



changed:

- all links to http://search.cpan.org in the GUI are changed to http://metacpan.org

- the fix in build 18204 to solve a shutdown problem on windows 2008R2 and windows 7 64-bit after installing KB4338818,KB4339093,KB4340556 (July 2018) is removed
  the issue was caused by the running DNS-server, which has opened 5000+ UDP ports at startup (this can be fixed using the command: dnscmd /Config /SocketPoolSize 500) - and
  the system socket handler which was unable to terminate 'close wait' connections
  setting HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters DWORD32 TcpTimedWaitDelay=60 (seconds decimal - 240 is default) may help




2018-07-23
fixed in assp 2.6.2 *Fortress* build 18204:

- fixes a shutdown problem on windows 2008R2 and windows 7 64-bit after installing KB4338818,KB4339093,KB4340556 (July 2018)

changed:
 
- ASSP_AFC 4.83 now scans the MIME header for viruses (possibly used by some UNOFFICIAL clamav signatures)




2018-07-19
fixed in assp 2.6.2 *Fortress* build 18200:

- if DMARC records were stored and 'DMARCReportFrom' was unset after that, wrong DMARC reports were created

- mail were blocked by the FileScan virus check, if the 'FileScanDir' was not available - in this case the check is skipped now

- if the 'NORUN' directive is defined for 'FileScanCmd', assp will wait some more time (max 10 seconds) for the scan process to be finished, if
  the content to scan is very large ( >5 MB ). 


changed:

- 'ccSpamNeverRe - Do Not Copy Spam Regex' now also belongs to scoring- and blocking reasons, as well as all headers added by assp (X-ASSP-..:)


added:

'BlockResendLinkOnly','User which get a ResendLink only *'
 List of users and domains that will get a ResendLink. If defined, only users listed here will get a ResendLink! 
 Using Groups is supported.'


'BlockResendLinkNo','User which not get any ResendLink *'
  'List of users and domains that will not get any ResendLink. If defined, users listed here will not get a ResendLink!
  Using Groups is supported.'





2018-06-30
fixed in assp 2.6.2 *Fortress* build 18181:

- the group-edit and group-show button were not available, if a group was defined in 'blockReportFile' 

- 'BlockResendLinkLeft' and 'BlockResendLinkRight' were not working for groups used in 'blockReportFile'

- defined include file loops caused stucking MainThread and endless assp restarts

- in some cases the timeline of debug output lines was broken


changed:

- a warning is shown in the maillog.txt if a defined regular expression matches an empty string

- a warning is shown in the maillog.txt if a defined regular expression matches any string




2018-06-23
fixed in assp 2.6.2 *Fortress* build 18174:

- the file defined in 'blockReportFile' was unexpected rewritten at blockreport creation

- changes to regular expression populated by the GlobalPenaltyBox-Server were some times not included in to the local files



2018-06-18
fixed in assp 2.6.2 *Fortress* build 18169:

- If a blockreport was requested for a 'Group' ([group@domain.org]) without specifying the recipent of the report (e.g. [group@domain.org] or [group@domain.org]=>*) - no report was generated

- If a schedule was changed for a blockreport, some times a high CPU usage was caused by the MaintThread or a software exception occured

- the time order of the loglines written to maillog.txt and SYSLOG was sometimes incorrect

- using two different (nonssl,ssl) listeners for webAdminPort (e.g.: SSL:55556|55555 ) was not working


changed:

- ASSP_AFC 4.82 is now able to extract OLE files and to analyze their content. This option can be confured using the attachment blocking exception rule

 :HLMSOLE - (HarmLess) Microsoft Office Compound File Binary (OLE) - MSOLE, except it contains forbidden files (the OLE::Storage_Lite module in PERL is needed)

- the file 'notes/loaded_perl_modules.txt' is more readable



2018-05-29
fixed in assp 2.6.2 *Fortress* build 18148:


changed:

- admins are now able to remove global and domain based entries from whitelist using the emailinterface
  admins are now able to delete global and domain based database records from the whitelistdb using the emailinterface
  admins are now able to delete global and domain based database records from the whitelistdb using the webinterface (GUI)

'EmailWhitelistRemove','Remove from Whitelist Address'
  'Any mail sent by local/authenticated users to this username will be interpreted as a request to remove addresses from the whitelist. Do not put the full address here, just the user part.
  For example: asspnotwhite
  
  EmailAdmins and EmailAdminReportsTo are able to force global or domain based removal and deletion requests as well as removal and deletion requests for other users - like:
  
  sender@domain.org,*
  sender@domain.org,@localdomain.org
  sender@domain.org,otheruser@localdomain.org
  
  Per default a removal request is processed. To delete records from whitelistdb, write "delete" (unquoted) into the subject of the report mail.
  NOTICE: removing whitelist entries will mark the records as NOT whitelisted!
  NOTICE: deleting whitelist entries will DELETE ALL related records! For example: an emailaddress is globaly whitelisted but markted as not whitelisted for a specific domain.
  Now if you DELETE the domain based record, all domain related records will be deleted - but because of the global whitelisting,
  all emailaddresses from this domain are now treated as whitelisted!





2018-05-17
fixed in assp 2.6.2 *Fortress* build 18137:

- ASSP_RSSSelectCode was not encrypted

- webSSLRequireClientCert was not working like expected

- unexpected behavior or exception, if a schedule was configured while any scheduled task was running



2018-05-11
fixed in assp 2.6.2 *Fortress* build 18131:

- the service ID was not shown correctly in the GUI on nix systems

- ASSP_AFC 4.81 fixes a perl crash : Can't locate object method "name" via package "Email::MIME"

- ASSP_RSS 1.05 fixes: links were not working in some RSS feed readers



2018-05-08
fixed in assp 2.6.1 *Fortress* build 18128:

- HMM was not working in build 18120, if 'DoPrivatSpamdb' was set to NO


2018-05-03
fixed in assp 2.6.2 *Fortress* build 18123:

- Running multiple assp instances on a single windows host as windows service failed, because all instances used the same service name. The GUI now shows the instance name at the top.

- The error message "Error: DKIM-cfg - no configuration left for any domain" was shown at startup, even genDKIM and genARC were both not selected.



2018-04-29
fixed in assp 2.6.2 *Fortress* build 18119:

- performance improvement for the rebuildspamdb task if pravacy levels are used for Bayesian and HMM

- disclaimer removal statistic output for the rebuild task

- using the analyzer web interface no longer prevents the MainThread from getting new SMTP connections



2018-04-27
fixed in assp 2.6.2 *Fortress* build 18117:


- changes in build 18112 and 18114 caused bombDataRe to check also the header of an email



2018-04-24
fixed in assp 2.6.2 *Fortress* build 18114:

- this release contains a significant performance improvement for the Bayesian and HMM checks - and the rebuildspamdb task



2018-04-22
fixed in assp 2.6.2 *Fortress* build 18112:

- this release contains a significant performance improvement for unicode character processing
  
- this release contains a significant performance improvement for regular expression processing


changed:

- DoNoFrom now also checks for RFC822 / RFC 1522 compliant email addresses and a check for a valid TLD is included



2018-04-20
fixed in assp 2.6.2 *Fortress* build 18110:

- certificates that provides the email address in SAN, were not accepted by ASSP_AFC for SMIME signing
  ASSP_AFC.pm is updated to version 4.80

changed:

- if ARC signing is enabled, assp will add a ARC-signature to reports and notification

- improved handling of DKIMWLAddresses and DKIMNPAddresses

- improved handling of Authentication Results provided by a trustedAuthForwarders host

- the default DKIM configuration file dkim/dkimconfig.txt is updated to version 1.02



2018-04-17
fixed in assp 2.6.2 *Fortress* build 18107:

- ASSP_AFC has left open a filehandle, which may cause an "too many open filehandles" exception
  version 7.49 fixes this problem

- the MIME header parsing in severa features for two equal MIME header tags was incomplete, if the two headers occured directly after each other

- IPv6 addresses were not exported by ExportExtreme


added:

'genARC','Generate and Add Authenticated Received Chain (ARC) signatures to all messages'
 If selected, ASSP will add "http://arc-spec.org" Authenticated Received Chain (ARC) signatures to all messages, if it finds a valid DKIM configuration in DKIMgenConfig for the sending domain.
 This will also be done for noprocessing mails. If available, the check results for SPF, DKIM and DMARC will be provided in the generated ARC-signature.
 This requires an installed Mail::DKIM"module in PERL.

'DoARC','Validate Authenticated Received Chain (ARC) Signatures'
  'If enabled, "http://arc-spec.org" Authenticated Received Chain (ARC) signed Mails are checked for the right signature sequence and contents.
  ASSP will show the ARC results and will trust the provided Authenticated Results for DKIM, SPF and DMARC if the signing host/domain matches 'trustedAuthForwarders'.
  This requires an installed Mail::DKIM::Verifier module in PERL.


changed:

- the FromStrict and the DomainIP check are moved after the SenderBase check.


'DKIMgenConfig','The File with the DKIM and ARC configurations*',
 The file that contains the DKIM and ARC configuration. A description how to configure DKIM, DomainKey and ARC could be found in the default file dkim/dkimconfig.txt.

'DoNoSpoofing4From','Do NoSpoofing for from:'
  'Do the NoSpoofing check also for header 'from:', 'sender:', 'reply-to:' and 'errors-to:' addresses.

'DoNoFrom','Check for Existing and Valid From: and Sender: Header Tag and Address',
  'If enabled, the MIME header is checked for valid From: and Sender: header tags.
  This header check fails and faults are counted, if both headers (From: and Sender:) are missing - or if any of these headers contains not a valid email address - 
  or if multiple of the same headers are found.
  The scoring value nofromValencePB is added for each detected fault.

'DoNoFromWL','Do DoNoFrom for Whitelisted'
  'Check for existing From: or Sender: header and address for whitelisted emails.

'DoNoFromNP','Do DoNoFrom for NoProcessing'
  'Check for existing From: or Sender: header and address for noprocessing emails.

'trustedAuthForwarders','X-Original-Authentication-Results and Authenticated Received Chain(ARC) Trusted Forwarder*'
....
 If DoARC is enabled and a host match is found for the most recent "http://arc-spec.org" Authenticated-Received-Chain(ARC)-Signature instance (highest instance number),
 the SPF-check, the DKIM-check and the DMARC-check will fully trust the provided ARC results.
....

'RunRebuildNow','Run RebuildSpamdb now'
....
  An real problem may become disclaimers and privat and corporate signatues. They are always added to outgoing mails, but also to local mails and reports.
  They can be found in most of the answers to your mails. And for example, they may be added by spammers to there spam mails - trying to fake good mails.
  Nobody can say, how the occurrence of such a disclaimer will affect the HMM and Bayesian results. It may possible, that these results differs from day to day, or block good mails, or let spam pass.
  The only way to prevent such results, is to remove the disclaimers, before the rebuildspamdb task builds the spamdb and HMMdb.
  To tell assp, which are your disclaimers, open the file files/disclaimer.txt using the "disclaimer definition" button below and put the disclaimers in to this file,
  the same way they are shown in your mail client. If you want to define multiple disclaimers, separate them by a line with a single dot.
  Lines in this file starting with an "#" are considered a comment, empty lines are ignored. ASSP will build a regular expression to identify and remove the disclaimers.
  
  example:
  
  # a comment
  your first disclaimers first line here
  your first disclaimers second line here
  .
  # also a comment
  your second disclaimers first line here
  your second disclaimers second line here
  
  This file will only be read at the rebuild task start. The resulting regular expression is written to "files/optRE/disclaimer.txt"
....





2018-04-13
fixed in assp 2.6.2 *Fortress* build 18103:

- Depending on the configuration and the installed/enabled perl modules, the HMM check and the Bayesian check may caused an "malformed UTF-8 character" exception, followed by a SMTP-worker restart.



2018-04-12
fixed in assp 2.6.2 *Fortress* build 18102:


- **** IMPORTANT ****  an unhandled exception caused very high CPU usage
 this BUG is in the code since assp 2.3.4 13156 (I'm sorry)




2018-04-10
fixed in assp 2.6.2 *Fortress* build 18100:


- the analyzer now prevents duplicate feature matching lines

- because the DKIM check was skipped, if assp has changed or removed headerlines, DKIMWLAddresses and DKIMNPAddresses was not working in every case

- myNameAlso was synchronized, this is no longer the case

- on some OS the 'autoflush' was not working for rebuild output files


changed:

- a regular expression containing the values of myName and myNameAlso is added to 'trustedAuthForwarders' every time (if myName is not set to ASSP.nospam)

- the recommended version of Mail::DKIM::Verifier (Mail::DKIM) is changed to 0.50


2018-04-04
fixed in assp 2.6.2 *Fortress* build 18094:

- the scheduled blockreport design was still broken, if no blocked mail was found

- if a very short time range (eg. less than 5 minutes) was defined for a statistic graph, an "modulus by 0" exception caused a mainthread crash


added:

'trustedAuthForwarders','X-Original-Authentication-Results Trusted Forwarder*'
 If an email contains a valid DKIM signature and the signature protects the "X-Original-Authentication-Results" header line in its h= tag (RFC7601) and the host in this header line matches
 this regular expression, DMARC will fully trust the provided original authentication results for SPF and DKIM.
 For example:  mx\d*\.domain\.com or ^2\.2\.2\.2$'


changed:

- images\svg.js (images.zip) is updated to version 1.04 - the click on a statistical graph now shows also the date (not only the time)

- for whitelist modifications and reports using the email-interface, the 'WhitelistPrivacyLevel' states (global,domain,privat) are show in addition to prevent confusion

- if hash data are shown in the GUI-Edit dialog, a sort (up/down ward) option is available


2018-03-28
fixed in assp 2.6.2 *Fortress* build 18087:

- the 'userAttach' functional enhancement (build 18085) was not recognized by the mail analyzer



2018-03-26
fixed in assp 2.6.2 *Fortress* build 18085:

- If an emailaddress, that contains a '+' was used in any address list matching parameter, assp never found a match, because the '+' was missinterpreted as a regular expression quantifier.

- enhance logging for 'enhanced Originated IP detection'

- the blockreport design was broken, if no blocked mail was found



changed:

- The default value for 'PenaltyError' is changed to '554 5.7.1 Error, send your mail to postmaster@LOCALDOMAIN to ensure delivery'.


- 'userAttach' got a functional enhancement
  ASSP_AFC.pm is updated to version 4.78 to support the same behavior.
  ...
  It may possible, that you want assp to deliver mails sent from a specific domain or emailaddress any way (without an attachment check). For security reasons this behavior can be only forced,
  if the sender was validated by SPF and/or DKIM and/or SMIME/PGP (Sig). The check is done by assp at runtime (mail processing) only!
  The definition described below must be done sepately for evey "good","block" as well as "zip" tag, for which the attachment check should be skipped.
  The (not case sensitive) definition tag starts with NoCheckIf= , followed by at least one state of "spf","dkim" or "sig".
  These states can be AND combined by writing them simply together like SpfDkim or SpfDkimSig in one word. To combine them in an OR logic, separate them by a dot like: Spf.Dkim .
  An combination for OR - AND would be: Spf.DkimSig . Whitespaces are not allowed in a NoCheckIf= definition!
  
  spf - the mail passed the SPF check - Notice: to validate against IP addresses for non SPF domains, use SPFoverride
  dkim - the mail is DKIM signed and passed the DKIM check
  sig - the mail contains a valid SMIME or PGP signature
  
  examples:
  ~~allowSDSSIn=>good-in=>NoCheckIf=SpfDkim.SpfSig,block-in=>NoCheckIf=SpfDkim.SpfSig
  sender@domain.org=>~~allowSDSSIn
  or
  sender@domain.org=>good-in=>NoCheckIf=SpfDkim.SpfSig,block-in=>NoCheckIf=SpfDkim.SpfSig
  which means: for sender@domain.org (sender) the good and the block check will be skipped, if the mail is SPF checked and DKIM validated - or the mail is SPF checked and has a SMIME/PGP signature.
  
  *@domain.org=>good-in=>NoCheckIf=Dkim.Sig,block-in=>NoCheckIf=Dkim.Sig
  which means: for the sending domain @domain.org the good and the block check will be skipped, if the mail is DKIM validated or has a SMIME/PGP signature.
  ...
  

- The template file 'dkim/dkimconfig.txt' is updated to version 1.01 to describe additionally DKIM settings

  ...
  # Inside the selector section you can define any supported value. Please read RFC 4871 or the documentation of the Perl module
  # Mail::DKIM to findout what values are for!
  #
  # For example:
  #    Identity=EMAILADDRESS
  #    Timestamp=0
  #    Expiration=86400
  #
  # The following replacement will be done by assp in every defined value:
  # The litteral DOMAIN will be replaced by the senders domain part.
  # The litteral USER will be replaced by the senders user part.
  # The litteral EMAILADDRESS will be replaced by the senders emailaddress.
  # The current time will be added at runtime to the values defined for Timestamp and Expiration, The values have to be defined in seconds!
  ...  


- An better example for Microsoft Exchange (AD) is added to 'LDAPFilter'

  ...
  or (eg. AD/Exchange 2013/2016)
 (&(|(|(|(|(&(objectclass=user)(objectcategory=person))(objectcategory=group))(objectclass=publicfolder))(!(objectclass=contact)))(objectclass=msExchDynamicDistributionList))(proxyaddresses=smtp:EMAILADDRESS)(!(msExchHideFromAddressLists=TRUE)))



- If 'AddDKIMHeader' is set to ON, the following X-ASSP- header lines will be added to incoming emails and .eml files:
  
  X-ASSP-DKIMidentity: IDENTITY-STRING
  X-ASSP-DKIM-FlagState: [whitelisted][, noprocessing]




2018-03-06
fixed in assp 2.6.2 *Fortress* build 18065:


added:

'DKIMWLAddresses','Whitelist these Addresses for valid DKIM Signature *'
 'If a valid DKIM or DomainKey signature is found and the signature identity (e.g. i=user@domain.tld) matches any of these addresses, the mail will be processed as Whitelisted.
  Note this matches the end of the identity address, so if you don't want to match subdomains then include the @. Note that example.com would also match spamexample.com but .example.com
  won't match example.com. Wildcards are supported. For example: sourceforge.net|group*@google.com|.example.com
  It is possible to make this check recipient dependend (eg: on a set of local domains and/or local users). Use wildcards (* and ?) to define domains.
  
  Use the following syntax to do this:
  
  *@anydomain=>*@any_local_domain - for domain to domain
  *@*.anydomain=>*@any_local_domain - for any sub-domain to domain
  user@anydomain=>*@*.any_local_domain - for user to any sub-domain
  It is possible to define more than one entry at the left and the right side of the definition (=>), like:
  *@anydomain|*@other_domain=>*@any_local_domain|*@other_local_domain - always separate multiple entries by pipes
  
  It is also possible to use a GroupDefinition in any or both sides, like:
  [identitygroup]=>[recipientgroup]
  [identitygroup1]|[identitygroup2]|*@domain=>[recipientgroup1]|[recipientgroup2]|user@local_domain
  
  NOTICE - that the local email addresses and domains are not checked to be local once.
  To define special characters like '* and ?' - use their hexadecimal regex representation like '\x2A and \x3F'.'


'DKIMNPAddresses','Noprocessing these Addresses for valid DKIM Signature *'
 'If a valid DKIM or DomainKey signature is found and the signature identity (i=user@domain.tld) matches any of these addresses, the mail will be processed as NoProcessing.
  Note this matches the end of the identity address, so if you don't want to match subdomains then include the @. Note that example.com would also match spamexample.com but .example.com
  won't match example.com. Wildcards are supported. For example: sourceforge.net|group*@google.com|.example.com
  It is possible to make this check recipient dependend (eg: on a set of local domains and/or local users). Use wildcards (* and ?) to define domains.
  
  Use the following syntax to do this:
  
  *@anydomain=>*@any_local_domain - for domain to domain
  *@*.anydomain=>*@any_local_domain - for any sub-domain to domain
  user@anydomain=>*@*.any_local_domain - for user to any sub-domain
  It is possible to define more than one entry at the left and the right side of the definition (=>), like:
  *@anydomain|*@other_domain=>*@any_local_domain|*@other_local_domain - always separate multiple entries by pipes
  
  It is also possible to use a GroupDefinition in any or both sides, like:
  [identitygroup]=>[recipientgroup]
  [identitygroup1]|[identitygroup2]|*@domain=>[recipientgroup1]|[recipientgroup2]|user@local_domain
  
  NOTICE - that the local email addresses and domains are not checked to be local once.
  To define special characters like '* and ?' - use their hexadecimal regex representation like '\x2A and \x3F'.'





2018-03-05
fixed in assp 2.6.2 *Fortress* build 18064:


changed:

- 'DoDomainCheck','Validate MX or A Record' is changed
  'If activated, the sender address and each address found in the following header lines ....
   ....
   If only an IP-address is found for a MX, the A record check fails, if the IP has no valid PTR and DoInvalidPTR is enabled.


- The analyzer now shows all log lines produced while the analyze task is running.

- The DKIM check now fails, if the signature_reject_reason is 'public key: not available'



added:

- the hidden config parameter 'AllowCodeInRegex' is added to enable the code execution in regular expressions

$AllowCodeInRegex = 0;   # (0/1) allow the usage of executable perl code (?{code_to_run}) in regular expression - change this ONLY, if you really know what you do


- 'SSLAdvancedServerConfigFile','File with Advanced SSL-Server Parameters'
  'Full path to the text file containing the server's advanced SSL parameters.
  If your SSL-server configuration requires additionally SSL-parameters according to IO::Socket::SSL and/or Net::SSLeay (for example: special Elliptic-Curve Diffie-Hellmann Key Exchange)
  and you don't want to use SSLWEBConfigure , SSLSTATConfigure , SSLSMTPConfigure confuration options, you may define a text file with your parameters here.
  NOTICE: assp will not check, if your configuration settings, made in this file, are valid - they are used as defined. In doubt, use SSLDEBUG to trace their effects.
  The settings in this file are passed as part of the IO::Socket::SSL configuration HASH to IO::Socket::SSL as they are defined.
  Any setting redefined in this file will override default internal assp settings as well as the above assp SSL configuration settings.
  The assp SSL settings below this tag are not effected.

  The syntax in this this file is the same like a HASH definition in Perl:
  - lines starting with an # are comments and are ignored
  - empty lines are ignored
  - each definition for a parameter has to be terminated with a comma
  - keyword and value have to be separated with =>

  example:
  # this is my special Elliptic-Curve Diffie-Hellmann Key Exchange for all listeners
  SSL_dh_file => full_path_to_your_DH-File,
  SSL_ecdh_curve => secp384r1,
  next-key => {
    subkey1 => subvalue1,
    subkey2 => [ARRAY-item-0, ARRAY-item-1, ...],
    subkey3 => {
      key => value,
      ...,
    },
    ...,
  },
  ...,
  last-key =>last-value,

  The defined file is watched for changes by assp. An possible reread of this file is only shown if SSLDEBUG is set to ON.
  It is highly recommended to read the documentation of IO::Socket::SSL and/or Net::SSLeay!
  Because the location of this file can be outside the assp folder, it can't be modified using assp! Please use an external file editor.




2018-02-28
fixed in assp 2.6.2 *Fortress* build 18059:

- improved HMM processing and detection for small emails



changed:

- if CCchangeMSGDate is used, only the value for the seconds will be randomly changed



2018-01-28
fixed in assp 2.6.2 *Fortress* build 18028:

- downloading files using HTTPS may caused a SEGV exception, if the download server has supported OCSP

- certificate validation may has been failed if 'proxyserver' was configured

- assp was possibly crashing, if there was no module available that implements the 'sha_hex' function



added:

- the hidden variable 'importDBShowProgress' is implemented
  setting this variabe to zero or 'undef', prevents the progess lines from the database-import function, to be written to maillog.txt - default is 1 


2018-01-18
fixed in assp 2.6.2 *Fortress* build 18018:

- workaround for a Regexp::Optimizer bug, where some times \r plus quantifier in a regular expression is optimized as \r\? - in case assp skips the optimization

- warnings and errors caused by the DKIM configuration are now shown in the GUI

- MaxAllowedDups may caused stucking workers in case of a havy workload

- a not installed perl module Thread::State caused exceptions and restarted high workers - the rebuildspamdb task was not running


- this version is released as fixup version 2.6.1 build 18022 on the sourceforge file download page


2018-01-18
fixed in assp 2.6.2 *Fortress* build 18018:

- 'MaxAllowedDups' was not working correctly under permanent havy workload conditions

- 'noCollectRe' was ignored, if a suspicious virus was found in the stored .eml file



added:

'MSGIDsigProc','Process valid Message-ID Signed Mails','0:normal|1:whitelisted|2:noprocessing|3:whitelisted and noprocessing',
 'How are received mails processed, if they contain a valid local MessageID-Signature/Tag (eg. because it is an answer/reply to a tagged mail).
 The default value is 'whitelisted'. Notice that noprocessing and/or whitelisted may prevent those mails from being collected in the corpus folders - check 
 noProcessingLog and NonSpamLog.'





2018-01-08
fixed in assp 2.6.2 *Fortress* build 18008:

- on some systems the maillog.txt OS buffer was not flushed and so the file tail was not readable just in time

- the perl garbage collection for the rebuild spamdb task was not working, if 'useDB4Rebuild' was not enabled

- if the very important perl module 'LWP::Protocol::https' is not installed, assp will try the installation at startup
  ******************************************************************************************************
  ************  IF THIS IS NOT POSSIBLE - install the module LWP::Protocol::https manually! ************
  ******************************************************************************************************
  nix:
  >cpan LWP::Protocol::https

  windows:
   active perl:
  >ppm install LWP-Protocol-https
  
   strawberry perl: (should normaly already include this module)
  >cpan LWP::Protocol::https 
  

- possibly unwanted X-ASSP passing reason hearders were added to outgoing mails - this is no longer the case

- the 'valid_Message-ID_signature' check has overwritten other whitelisting and noprocessing reasons, this caused reply mails stored outside the corpus





2017-12-28
fixed in assp 2.6.2 *Fortress* build 17362:

- SMIME signing was completely skipped for a defined sender address, if a recipient exception (eg. rcpt=-recipient@domain.com) was defined for this sender address
  this issue is fixed in ASSP_AFC 4.77
  

- Changing the certificates, KEY-password and RSA-privat-keys caused unexpected errors written to the maillog.txt for each defined SSL-listener
  until all required parameters were changed by assp in a sequence.
  The renew of the SSL-listeners and SSL-context is now delayed until all required SSL-parameters are changed.
  



2017-12-21
fixed in assp 2.6.2 *Fortress* build 17355:

- public released  2.5.6 build 17355

2017-12-18
fixed in assp 2.5.6 *Fortress* build 17352:

- a memory leak in report connection handling is fixed

- ASSP_AFC: non ASCII characters in 
  'ASSP_AFCReplBadAttachText','Replace Bad Attachments Text'
  and
  'ASSP_AFCReplViriPartsText','Replace Virus Parts Text'
  were wrong MIME-encoded in the delivered mail

changed:

- 'MaxAUTHErrors','Max Number of AUTHentication Errors'
....
If your MTA offers AUTH without supporting it (has no user accounts) define a negative value here (e.g. -1).
In this case assp and the MTA will function as an AUTH-honeypot, the peer will get an penalty at the first AUTH request.
....

- in case a blocked attachment mail is resent in block reports by admins, the '[no] scan' option is added to the request
  mail body

- ASSP_AFC version 4.75 fixes a minor logging output issue

- ASSP_AFC version 4.75 has a new function for VBA detection. To enable it set
  $ASSP_AFC::VBAcheck = 0;     # enable(1)/disable(0) the executable VBA script check
  to '1'


2017-12-07
fixed in assp 2.5.6 *Fortress* build 17341:

changed:

- improved connection data cleanup in idle mode (sleeping threads)

- typo corrections in the GUI



2017-12-05
fixed in assp 2.5.6 *Fortress* build 17339:


changed:

- It is now possible to define a SSL-listener for all listeners (SMTP, WEB, STAT) - to do this write SSL: in front of the listener definition.
  Examples:
  225
  SSL:325
  225|SSL:325
  127.0.0.1:225
  192.168.1.1:225|192.168.2.1:225|SSL:192.168.1:325

- improved error handling for the new SSL code in build 17338



2017-12-04
fixed in assp 2.5.6 *Fortress* build 17338:

- orphaned connection data and handles may caused errors like:
  - too many open files
  - invalid filehandle
  - ....
  It was also possible, that the connection screen and the worker status screen have shown incorrect data.
  On some systems those orphaned connections caused unexpected high CPU and memory usage.

changed:

- 'maxSSLRenegotiations' is only checked for incoming mails, not for local and outgoing mails

- the default value for 'maxSSLRenegotiations','Maximum Allowed SMTP SSL Client-Initiated-Renegotiations' is changed to 10

- SSLDEBUG now writes the debug information of the SSL handling to maillog.txt

- For all SSL listeners and STARTTLS connections where assp acts as server and all SSL/TLS connections to defined destinations only a single SSL-Context is used
  for each peer to speedup the SSL connections and to reduce memory usage.
  Setting the hidden configuration parameter 'enablePermanentSSLContext' to zero or 'undef' will force the old behavior (create and delete the SSL-Context for each connection).
  Setting the hidden configuration parameter 'enablePermanentSSLContext' to zero or 'undef' is not related to SNI configurations. For SNI server configurations a permanent
  SSL-Context is used every time.

- improved SNI support for environments with a large amount of SSL-certificates and keys

- speed improvement of the rebuildspamdb task

added:

- ASSP_AFC version 4.74 is able to check for 'Microsoft Office Compound File Binary (OLE)' attachments - the exception tag is :MSOLE

- The above referenced hidden configuration variable 'enablePermanentSSLContext' is added  
  our $enablePermanentSSLContext = 1;      # (0/1) enable usage of permanent SSL Context - maxunused = 1 hour, max lifetime = 1 day (default = 1)
  If set, assp will reuse an available SSL-Context unitl this context is not older than one day or it was unused for over one hour.



2017-11-19
fixed in assp 2.5.6 *Fortress* build 17323:

- on 'apply changes', google chrome (v57 and higher) may has thrown an error about 'x-xss-protection', if the GUI in http mode was used by user 'root'

- it was possible that setting 'FileScanCMD' to 'NORUN' caused stucking workers, if the online filesystem virus scanner detected a virus and locked the checked
  file permanently
  
- ClamAV and the FileScanner were called on no content (zero bytes), which sometimes caused an unexpected virus detection or a wrong content replacement by ASSP_AFC

 
- ASSP_AFC version 4.72 fixes a logging mistake

  

2017-11-13
fixed in assp 2.5.6 *Fortress* build 17317:

changed:

- If the filename for an attachment contains no extension, but the given Content-Type MIME header provides the filetype, the related extension is added
  to the file name to prevent unexpected blocked attachments.

  related to this change, the ASSP_AFC.pm Plugin is updated to version 4.71
  
  

2017-11-06
fixed in assp 2.5.6 *Fortress* build 17310:

- If 'StoreCompleteMail' was set to 'disabled'(0), the stored corpus files (.eml) were smaller than the value defined for 'MaxBytes'.


2017-11-03
fixed in assp 2.5.6 *Fortress* build 17307:

- switching the effective and real UID on BSD based OS may have been failed

- If ASSP was unable to accept a client socket connection, a retry may has caused a SEGV on some OS. The retry is now skipped.

- The output of the used UserAttach ZIP: regular expression in the analyzer is now normalized.



changed:

- The default value of 'tlsValencePB' 'OK, Is a SSL/TLS connection, default=0 +' is changed from -10 to zero.

- IP's with AUTH errors, faked AUTH-errors and SSL-renegotiation attacks are now reported to the Grip-list server

- Notification emails are now showing the matched log-text and the used regular expression at the end of the email.


added:

- To prevent DoS attacks in SSL renegotiations the hidden configuration variable 'maxSSLRenegDuration' is added - the default value is 10 seconds.
# the SSL/TLS renegotiation counter will be reset after this number of seconds without a renegotiation request and any regular data are sent or received
our $maxSSLRenegDuration = 10;


'maxSSLRenegotiations','Maximum Allowed SMTP SSL Client-Initiated-Renegotiations'
 'Maxumum count of allowed SSL/TLS client initiated renegotiations to prevent DoS.
 If this count is exceeded in a connection within 10 seconds, the connection is terminated, the connected IP is registered in banFailedSSLIP and new connections
 from this IP address are rejected for 15-30 minutes. An IP-Score of PenaltyExtreme but at least 150 is used for the IP address.
 Zero disables this feature - default is : 2 attempts.'



2017-10-24
fixed in assp 2.5.6 *Fortress* build 17297:

- upper case UserAttach templates caused an exception in attachment processing
  ASSP_AFC 4.70 is required too, to fix this issue
  NOTICE: build 17297 is at least required to use ASSP_AFC version 4.70 !
    


2017-10-08
fixed in assp 2.5.6 *Fortress* build 17281:

- active connection counting and limiting was not working if clients/servers used the STARTTLS command

- analyzing the memory usage in the threads cause a SEGV in 'Devel::Size' on newer perl versions - this feature is deactivated in the code permanently


changed:

- Because sourceforge is discontinuing the CVS support, the versioning support for the ASSP development is sitched to SVN.
  Starting with this build the download location is changed to:
  
  https://sourceforge.net/p/assp/svn/HEAD/tree/assp2/trunk
  
  The folder structure and file location in SVN is the same like it was in CVS.
  
  Previouse versions (including V1) and builds are still available at CVS for download, as long as souceforge provides browser access to CVS.
  
  http://assp.cvs.sourceforge.net


2017-10-03
fixed in assp 2.5.6 *Fortress* build 17276:

changed:

- On systems which supports the thread priority settings, some time critical task are running faster (clean PBBlack, rebuildspamdb).


2017-10-02
fixed in assp 2.5.6 *Fortress* build 17275:

- On a secured Windows Server 2016, it was possible that the options to 'stop' and 'pause' the running ASSP service were not available (greyed out).
  In this case, it was also not possible to manage the service using 'sc' and 'net'.




2017-09-25
fixed in assp 2.5.6 *Fortress* build 17268:


- There were two options missing in the Archive::Libarchive::XS call in ASSP_AFC, which are not supported without a special definition per default.
  - raw compression formats
  - empty compressed files
 Both types caused an 'Unrecognized archive format' exception at decompression time and an additionally call to a second (or third) decompression engine (eg: 7zip)
 Version 4.65 fixes this issue - those files are now decompressed by Archive::Libarchive::XS, if they are supported by the module.

- it was possible, that the used SSL/TLS cipher was added multiple time to the assp 'Received:' MIME-header line
 

changed:

- improved MS exchange MTA detection - related to '$CCchangeMSGDate', see build 17261

- 'DoSameSubject' uses a similarity check instead of an equality check

- 'MaxAllowedDups' uses a similarity check instead of an equality check

- using ASSP_AFC 4.65, the version numbers of the archive modules are now shown in the Perl-modules status screen

- if Encode::Detect is installed, it will be used instead of Encode::Guess to provide better decoding results



2017-09-18
fixed in assp 2.5.6 *Fortress* build 17261:


- if 'STARTTLS' was used to resend a mail, the second 'EHLO' possibly used 'localhost.localdomain' as hostname (instead of 'myName'), which may caused problems on some MTA's,
  because they expect to get the same EHLO again - both EHLO commands are now using 'myName'


changed:

- the function of the hidden parameter '$CCchangeMSGDate' is enhanced

our $CCchangeMSGDate = 0;                ## (0..31) change the 'Date:' MIME-header on CCmail (sendHamInbound), ForwardSpam (sendAllSpam) and resend mail
                                         ## MS-Exchange may require this, because duplicate mails will be removed silently, if they contain an equal 'Date:...' MIME-header
                                         # bit 0 = 1 ( +1) -> set all bits (1 - 4) to 1 for backward compatibility ( same as 30 -> 2+4+8+16 )
                                         # bit 1 = 1 ( +2) -> force change at CCmail
                                         # bit 2 = 1 ( +4) -> force change at ForwardSpam
                                         # bit 3 = 1 ( +8) -> force change at resend mail
                                         # bit 4 = 1 (+16) -> general disable the automatic detection of a local MS-Exchange MTA by checking the SMTP banner / greeting
                                         ## The default is zero (0), which means: the 'Date:...' MIME-header is not forced to be changed in either case,
                                         ## but it will be changed, if a MS-Exchange MTA is detected using $ExchangeBannerRe against the SMTP banner / greeting.
                                         ## To disable this feature completely - set this value to 16.




2017-09-17
fixed in assp 2.5.6 *Fortress* build 17260:

- Depending on the used perl version the following chinese charsets were not supported by the perl module 'Encode', even the module 'Encode::HanExtra' was 
  installed:  big5plus , euc-tw , gb18030.
  These charsets are now registered to 'Encode' on NON-EBCDIC systems by assp. The installation of the perl module 'Encode::HanExtra' is mandatory to support these
  charsets.

- If an unknown (not registered to Encode) MIME-charset was found in an email, it was possible that the thread died unexpected throwing an UTF-8 fatal error.
  Those mail data are now processed binary, in rare cases the content of such a mail is ignored.


added:

- ASSP_AFC version 4.62 now supports the definiton of custom excutable checks. Special coding in lib/CorrectASSPcfg.pm is required.

our $SkipExeTags = [];  # customized skip tags ('CUST1','CUST2'...) for external executable checks defined in lib/CorrectASSPcfg.pm
                        # usage in 'UserAttach' : ':CUST1',':CUST2'
our $checkExeExternal;  # custom subroutine to check executables external (eg. lib/CorrectASSPcfg.pm) - $ASSP_AFC::checkExeExternal->($self,\$sk,\$buff,$raf,\$pdf) 
                        # if the internal check has not found an executable
                            # self - the ASSP_AFC object for this mail
                          # the following paramters are refences to scalars
                            # sk - active skip tags at runtime
                            # buff - up to first 64 binary bytes of the attachment
                            # raf - complete binary content of the attachment
                            # pdf - decoded binary PDF content, if the attachment is a PDF , otherwise undef

our $checkExeExternalForce; # same as $checkExeExternal - but called weather the internal check has found an executable or not - 
                            # $ASSP_AFC::checkExeExternalForce->($self,\$sk,\$buff,$raf,\$pdf,\$type)
                              # ....
                              # type - contains the previous detected executable type description or undef



2017-09-11
fixed in assp 2.5.6 *Fortress* build 17254:

changed:

- attachment blocking:
  - If a file extension regular expression is wrong defined as (1) '*' (leading asterix) or (2) '?' (leading question mark) the definition no longer fails.
    Instead the regular expression is now corrected to (1) '.*' and (2) '.?' and a waring is written to the maillog.txt.



2017-09-04
fixed in assp 2.5.6 *Fortress* build 17247:

- ASSP_AFC 4.61 is released

- if any of HTML parsers was selected, the modul was not shown in Module Stats screen


changed:

- an new exception switch is added to the 'UserAttach' function - ASSP_AFC 4.61 is required to provided this also for compressed attachments (zip:...)

description changes:
....
  Notice the leading -- in front of the --doc regular expression in the last example. The leading -- removes all occurences of this regular expression from the resulting entry, 
  here from "block-in" (NOT from block!) at configuration time. You would need to define --doc in the "block=>" entry as well, to remove such occurences there.
  Because the -- exceptions are processed at configuration time, such a definition will not overwrite an opposit rule definition: sender > recipient and recipient < sender
  - which are combined at runtime (attachment check).
  If you want assp to process such a "remove extension directive" at runtime (to make the recipient <> sender rule overwrite working for this address),
  use for example -+doc instead of --doc. Be carefull creating weak blocking rules using the -+ directive. Make sure the sender and recipient address can
  NOT be faked (eg. SPF-strict, DKIM)
  ASSP will resolve all extension regular expression templates and all rule tempates and will combine them all in to one resulting domain or user attachment rule.
  ASSP will throw a warning, if a rule template is define multipe times - like: *@domain.com=~~commonRule,~~devRule - here ~~devRule already contains ~~commonRule
  It may happen, that the resulting attachment rule contains one or more extension regular expressions multiple times - this is harmless and will be internaly corrected,
  but try to prevent it.
  
  This feature replaces all of the above level definitions (BadAttachL1 ....L2 ....), if at least one valid (not zip:... from the ASSP_AFC Plugin) attachment blocking 
  or allow rule is found for the envelope sender or the first envelope recipient of a mail!
  good, good-out and good-in - and also - block, block-out and block-in - will be logical OR (pipe '|') combined from the matched rule for the first envelope recipient 
  and the envelope sender - according to the mail flow.
  The defined blocking rules for the envelope sender and the first envelope recipient are than combined together using the same OR logic (pipe '|') at runtime.
  The attachment block rules for a specific email are looking as follows: (replace block with good to get the attachment good rules)
  incoming mail: recipient-block|recipient-block-in|sender-block|sender-block-in
  outgoing mail: sender-block|sender-block-out|recipient-block|recipient-block-out
....



added:

- In the 'config info' section of the left menu, a new link to 'Privat Config Notes' is available. It can be used for general notes and privat documentation.



2017-09-01
fixed in assp 2.5.6 *Fortress* build 17244:

- a domain name in an URL that starts with number and dash like "2017-", was wrong detected as strong obfuscated IP address

changed:

- the over a year existing hidden configuration parameter 'HTMLParser' is now changed to a regular configuration parameter

**** ATTENTION ****
 If you still use this hidden parameter, remove any related code from the startup script or CorrectASSPcfg.pm BEFORE you upgrade to this version!
 Configure the parameter after the first start in the GUI!
*******************

 'HTMLParser','Use this HTML Parser','0:buildin|1:HTML::Strip|2:HTML::TreeBuilder',
 
  Commonly HTML/XML is used in emails. The HTML/XML tags are too variable to use them for Bayesian- and Hidden Markov Model analysis.
  For this reason, these tags are removed from the HTML/XML content to get the clean text of the email.
  The assp buildin regular expression HTML-parser is now used for decades. It got large improvements over the time, how ever - the correctness is only 95%.
  But assp is able to use HTML::Strip or HTML::TreeBuilder, which are powerfull perl modules to parse HTML code nearly 100% correct.
  HTML::Strip and HTML::TreeBuilder are getting there best result, if the full HTML code is provided. In case you select any of the both modules,
  it is recommended to set MaxBytes to 50000 (be carefull on heavy load systems - spam bomb regular expressions will take longer using 50000!).
  HTML::Strip is the fastest module and the default setting, because it is written in C. If you can not install it, use the buildin or HTML::TreeBuilder.
  HTML::TreeBuilder is the slowest way to parse HTML code, the assp buildin processing is three times faster, HTML::Strip is five times faster than HTML::TreeBuilder.
  If you select any of the perl modules and this module is not installed, fails to load or it returns no content, assp falls back to the buildin code.

 **** Switching from the buildin HTML parser to HTML::Strip a 10% faster rebuildspamdb task is expected (for MaxBytes = 50000).
 **** HTML::Strip improves the word processing for Bayesian and HMM, because of a much better language detection in the word stemming engine.
 **** SpamBombs will work more accurate, if HTML::Strip is used

  To provide any of the perl modules HTML::Strip and HTML::TreeBuilder you need to install them using PPM or CPAN.
  The mod_inst.pl and mod_inst_ocr.pl got an update to version 2.03 to install both modules.
  
  ASSP_AFC.pm version 4.60 is available. It got an improvement by an 40 seconds timeout watchdog ($ASSP_AFC::maxProcessTime), to prevent stucking workers.
  

2017-08-31
fixed in assp 2.5.6 build 17243:

- if two email addresses were defined in the from: header tag - like: from: dummy@localdomain.com <sender@senderdomain.org>
  the first address was used by assp instead of the right second. This made spam detection difficult and caused the DKIM check to fail.
  
  
added:

- 'UserAttach' got an enhancement - it is now possible to define and use regular expression templates as well as rule templates
  - the GUI is changed
  .....
  block=> rules cause specific file types to be blocked (but does not block the others).
  good=> rules block all file types except for those specified in the rule.
  ....
  
  It is possible to define templates (see the preceding single tilde ~ ) for extension regular expression and to use them in any entry at any place
  (except other extension regular expression templates) - like:
  
  ~executables => cmd|com|cpl|exe|exe\-bin|lnk|pif
  ~scripts => js|pl|ps1?|sh|vb[es]?|wms|ws[cfh]
  user1@domain.tld => block => ~executables|~scripts|mht|ms[cipt] , block-in =>:MSOM , block-out => :CERTPDF
  [allDomains] => block => ~executables|:CSC
  
  Extension regular expression template names have to start with a single tilde. Allowed name characters are A-Z, a-z, 0-9 and underscrore.

  It is also possible to define rule templates and to use them in combination with any other rule definitions or rule templates.
  Rule templates starts with two tilde (~~template). Allowed name characters are A-Z, a-z, 0-9 and underscrore. For example:
  
  ~~commonRule=>block=>~executables|~scripts|xls,block-in=>:MSOM,block-out=>:CSC
  ~~devRule=>~~commonRule=>block-out=>:WIN|:ELF
  ~~allowALL=>good=>*
  *@domain.com=>~~commonRule
  [IT]=>~~devRule
  user@domain.com=>~~commonRule,~~anySecondRule,~~anyOtherRule=>block=>~anyExt,block-in=>~otherExt|xls|--doc
  
  Notice the leading -- in front of the --doc regular expression in the last example. The -- removes all occurences of this regular expression from the resulting entry,
  here from block-in.
  ASSP will resolve all extension regular expression templates and all rule tempates and will combine them all in to one resulting user attachment rule.
  ASSP will throw a warning, if a rule template is define multipe times - like: *@domain.com=~~commonRule,~~devRule - here ~~devRule already contains ~~commonRule
  It may happen, that the resulting attachment rule contains one or more extension reglar expressions multiple times - this is harmless, but try to prevent it.
  ....
  
  


2017-08-11
fixed in assp 2.5.6 build 17223:

- public released  2.5.6 build 17223


